Online archetype and template tools

Current archetype and template tools intended for clinical modelling do not support all new features in ADL 2.0 (previously called ADL 1.5). Some clinicians also have limited possibilities/permissions to install the current editing applications on their computers (hospital security rules etc). A more modular editing framework than the some of the current somewhat monolithic applications would also be desirable so that alternative implementations of parts/functions could be tested/used. The main mission of the currently starting project phase is to address these issues. After that initial phase, future development may further explore for example versioning and collaboration features.

The openEHR 2014 roadmap meeting agreed in principle on starting a community tooling project aiming to create an open source based web-accessible archetype and template editing framework/workbench capable of ADL 2.0. The meeting report http://www.openehr.org/news_events/2014-09-16_openehr_meeting_report states that:

A next generation archetype tooling project will be commenced.

• A technical group will develop an outline architecture and project proposal in the next 3 weeks.

• The project will be open source, and be open to use by developers from any organisation using archetypes.

• Most end users (e.g. clinicians) should be able to use main parts of the tooling via modern web-browsers, without needing to install extra software locally. (Some functionality might require access server-based components though.)

• We will aim for a combined project that covers the needs of openEHR, 13606, CIMI and any other archetype user community.

Some other (less edited) input from the Oslo meeting is available at http://www.openehr.org/wiki/display/oecom/September+2014+Roadmap+Meeting 

UPDATE Feb, 2017: Marand pre-announces Angular2-based new version

Pre-announcment of Angularr2-based version https://twitter.com/marandlab/status/826832144672686081 
(Today, 6 Feb 2017, the URL https://ehrscape.marand.si/designer/ seems to host a previous version. The version at https://github.com/openEHR/adl-designer is also a previous version)

UPDATE Aug 21, 2015:
Developer preview of web based archetype and template editors - RELEASED today at Medinfo2015 !!! 

Please view theutl slides from the release announcement: https://goo.gl/7Cd52R 
(Presented at the Medinfo 2015 openEHR developer workshop, Aug 21, in São Paulo, Brazil)

Link shortcuts: 

• Demo:
  • Note that this is a developer preview not yet suitable for clinician use-
  • http://ehrscape.marand.si/designer/template-editor.html
  • http://ehrscape.marand.si/designer/archetype-editor.html
  • Remember to click “load”, “new” or “specialize” to get started
• Source code:
  • https://github.com/openEHR/adl2-core

Many thanks go to Marand for doing a huge amount of design, coding and testing!

UPDATE Oct 23, 2014:

UPDATE Oct 20, 2014:

We had a second meeting Tuesday October 21. Notes and chat available at http://www.openehr.org/wiki/x/DoAiAw The sections "Roadmap" and "Actors/developers and their interests/contributions" were updated as a result of the meeting.

UPDATE Oct 14, 2014:

Recording, chat transcript etc. from the first online meeting is now available, see: http://www.openehr.org/wiki/x/O4AYAw 

Some major points from the first online meeting:

• Regarding "clinician-friendly" editor views, template editing was considered more urgent to get new tooling for than for archetype editing. Partly because of the state of current template editing options and partly because template authoring is a usecase that most clinicians are expected to meet more often than archetype authoring. 
• Vendors and others have an interest in generating (backwards-compatible) artifacts like ADL 1.4 archetypes, operational templates etc. The ADL workbench most likely supports that already today. Those depending on such conversions from ADL 2.0 artifacts to older formats should focus on testing this early during this first project phase. The workbench could possibly be updated to process such requests from web applications and thus work as a server component in a new editing framework. If this works, then the new tooling could focus on ADL 2.0 initially.
• There is a confusion regarding names and features of possible future versions of the 1.x family of ADL (currently at ADL 1.4) focusing on non-breaking changes.  
• There are some doubts regarding if a web/HTML5-based interface will be powerful enough, but most seem to consider it to be worth trying.
• Online collaboration features would be nice for initial mindmapping-phases of archetype development, but likely of less importance for later stages.
• All involved/interested should edit the project wikipage, at least the the part describing yourself under the heading "Actors/developers and their interests/contributions"
  
• Tom (below on the project wikipage) described the internal APIs of the ADL workbench as a source of inspiration
• ...feel free to add more here

Organisation, management and meetings

Erik Sundvall is responsible for arranging a first draft wikipage (this one) and an initial online meeting, but not necessarily the one coordinating the rest of the project. (An organization/person that can dedicate project leader time for some months would be welcome to volunteer.

A "doodle" where you can select time-preferences for 1-hour meetings is available at: http://doodle.com/gm456xavg3fcgvw5 (date alternatives ranging between Oct 14 to Oct 27) A first meeting is already finished. We might pick other times among these for followup meetings. 

Based on the doodle above (read on october 16) the most likely follow-up meeting time could be October 21 and/or October 23, the time would be the same both dates, at 12:00-13:00 Swedish (lunch) time 

Default meeting mechanism, if nothing else specified, very participant-scalable, but sometimes slightly lagging, Adobe Connect meeting room at https://connect.sunet.se/openEHR. Please try everything out before meetings so that you have audio (and possibly video) working properly. Ask a friend/colleague to log in to the room too and try talking to each other for a while. Use the audio setup wizard (see image below) to calibrate speaker/mic, it is somewhat hidden - you need to click the "meeting" button/meneu.

.

Having a physical meeting was also suggested and that will hopefully be possible later for at least some participants. But we start online to be better prepared for physical meetings. Online participation possibilities during parts of physical meetings will be seriously considered.

Wiki discussion-threads can be created below using the comment function of the wiki.

Possible mailing list discussions regarding this project should initially be directed to the openEHR implementers list (if the wiki comment system below is not useful enough). Major questions, events and news should be announced on the openEHR implementers list (since we do not expect everybody to monitor the wiki). If traffic gets to intense there consider separate list.

Roadmap

• First get a rough componentization up and running (modifyable AOM-tree datastructure)
  • something that can output ADL as one of the first serialization formats in order to feed that into a backend/serverside conversion chain (based on the ADL workbench to begin with)
  • as a start it is good if the tool can read from (and write to) Git repositories
  • make sure the solution is modular and extendable so that people can build different tools, UI on top of it.
    • Seref Arikan explaind it well in the chat: I'd like to see a bottom up roadmap. Some model driven core components, parsers, AOM etc. UI and functionality can go into 100 different directions
• We should try to have AE & TD replacements by the end of 2015. Get a framework with basic editing capabilities up and running before November 2015 (hopefully a lot sooner) otherwise consider ending this project and hope that other actors step in.
• Develop ADL 1.4 migration strategy for implementers.

Actors/developers and their interests/contributions

Please add full names, specific interests, possible contributions under your organisation/name and correct misunderstandings and other mistakes.

• Ocean Informatics - Thomas Beale, Seref Arikan, Sebastian, Peter Gummer

  • Component architecture of new tooling

  • Tom:

    • providing webservice/rest interface to OPT generation based on AWB?

    • single file template e.g. https://github.com/openEHR/adl-archetypes/blob/master/Example/openEHR/single_file_template/templates/openEHR-EHR-COMPOSITION.t_clinical_info_ds.v1.adls

    • OPT 1.4 and OPT 2.0 should be in formal specifications

• FreshEhr (Ian McNicoll)
  • Help validate ADL2.0 -> 1.4.OPT generation
  • Explore optimal repository organisation including integration with git/githib

• Marand - Bostjan, Borut, Marko

  • Contributed ANTLR grammar and Java implementation of ADL 2.0 parser, see https://github.com/openEHR/adl2-core (perhaps in a distant future also interested in a pure javascript parser)

  • Initially acively developing/exploring/"sandboxing" and figuring out possible AOM JSON structures and serialisations - Something will be available to the community to look at in approximately 2 weeks.

  • Exploring split of serverside- vs client/editor-side components

  • Interested in getting a template designer working

  • Interested in mapping mechanisms between "at" and "id" codes (1.4 vs 2.0)

• Code24 - Sebastian Iancu

  • Experience with PHP / Sencha ExtJs

  • In the future we can provide REST API services to make conversions or validations

• DIPS - Bjorn Naess

  • Use cases and improvement suggestions compared to current tools

  • Mostly developing on .NET but have a few web developers that can be engaged

• RaySearch - TBD

  • Interested in parser implementations using Javascript

  • ...

• Linköping University and the County Council of Östergötland: Erik Sundvall (openEHR software program coordinator)

  • Drafting first wikipage, setting up first online meeting, 

  • Announcing project news via mailinglist(s)

  • Help research and test "Operational Transformation" possibilities/frameworks (for undo/redo and collaboration)

  • Interest in helping out with GUI considerations (not lead GUI development)

  • Interest in pedagogical RM-visualization (at some later stage of project when more important things already work..

• University of Auckland - Koray Atalag & Aleksandar Zivaljevic

  • Annotation tools - formal "GUI Directives" (see details below)
    
  • ...
• Samuel Frade
  • Transformations e.g. OPT -> HTML etc (both HTML trees structures and more "clinically friendly" ones)
    
• Nousco -- Dong-Won Choi
  • We have focused on ADL 1.4 because of project. 
  • 1.4 to 2.0 and vice versal most impotrant issue but still impotant in support ADL 1.4. So Need to detailed Document about APIs or Libs to faliciltate tool development. Actually too lack of formal documents & codes or APIs about tool development resources in ADL 1.4.

  • and then I think it is right to move to ADL 2.0 (of course 2.0 is important issue !)
    

Initial/potential approaches identified as interesting during & after the roadmap meeting

• HTML5+Javascript based user interface (should work nicely in full screen mode too).

• Possibly server-side components (using various implementation technologies) for model conversion, validation and other things that are already (fully or partly) implemented. Existing components that already support ADL 2.0 are of course extra interesting. For offline use on not-locked-down-computers such "server components" can of course run locally.

• Angular JS for GUI to model mapping + Bootstrap for CSS, some components etc

• Operational Transformation (see explanations below) could provide both robust undo/redo/history functionality (and in the long run synchronus multi-user/multi-device editing).

• These tools might be desirable to integrate with different vendor solutions, so modular design and configurable appearance (via CSS?) is preferred.

• Use Apache 2 licence (default for new openEHR foundation hosted projects)

Proper modularisation will make it easier for different developers to work on (or replace) separate parts. Suggest modules and/or APIs below

Potential API:s/interfaces between modules

• directory that exposes/lists/fetches/saves archetypes/templates from/to CKM, Github or other sources
• a parsed modifiable tree-representation of AOM objects

  • probably as a tree of Javascript or JSON "objects"
    (TODO: What kind of trees/datamodels will parsers generate? Would a (partly handcoded?) Javascript-implementaton of the AOM be needed?)

  • a way for different implementations of validator functions to flag/annotate branches of the tree as valid/conformant or not.
  • a way to add/modify manually entered comments and discussions to branches (including things that may not get serialized in a final published archetype).
  • some kind of tree or branch that in addition to proper AOM objects can hold yet unstructured ideas or semi-structured notes (mindmaps, bullet lists etc) that can later be converted (manually with some automation help) to proper AOM structures. 
  • it would be nice if this whole tree could be versioned (and later shared) via OT (Operational Transformation)
  • a way to subscribe to change-notification of nodes
    (TODO: Does Angular JS or OT-libraries already provide what is needed for change-notification subscriptions?)
• a detailed representation of RM models (openEHR, ISO 13606, CIMI, etc) easily traversable and accessible for other modules
  • I believe BMM-files (see link below) are used as source in the ADL wokbench and in Link EHR I believe they use "RM-archetypes" in XML format as input (add link).
• ADL Workbench is a good source of some internal APIs:
  • 'archetype repository' - file system, including Git interface, to load archetypes from
  • 'archetype library' - in-memory (loaded) and indexed archetypes and templates - most of the rest of the tool works from this
    • includes statistics view
  • 'RM schema' - computational interface to loaded RM(s); there is a library of BMM classes that do this
  • Parser interface - component that performs parsing from ADL source
  • Validator interface - component that performs validation of archetype lineage / dependency graph
  • Flattener interface - component that performs flattening of archetype lineage; also dependency graph for templates
    

Potential modules

• • Clinician-friendly GUI-representations of the tree-parts (for example similar to different panes, subtrees and widgets in the current archetype editing tools)
  • Clinician-friendly form-generators that show something similar to what an entry form using the archetype/template will look like for an EHR end user.
  • Nerd-friendly GUI-representations of the tree (for example similar to some views in the current ADL workbench) 
  • Nerd-friendly text-representations of the tree (ADL, JSON, YAML, whatever...)
    • There are a lot of open source, highly configurable, text editor components out there, many support syntax-highlighting etc out of the box.
      Suggest editor examples and describe interesting features:
      • The inline editor widgets in http://brackets.io/ (for example color selectors) make an interesting hybrid between text-based and GUI-based editing
      • ...
  • Serializers
  • Parsers
  • Terminology binding tools
  • Annotation tools - since ADL 2.0 Templates are indeed Archetypes it'd be good to support current Template Designer's annotation capabilities. We are mostly interested in creating a formal "GUI Directives" section - can even be considered to be part of ADL in future once we know more about what is pure presentation specific (e.g. that can go as template level) as opposed to semantic (which should be part of archetype). More research is needed!
  • ...

Suggestions/ideas hard to classify/separate as above

• ...

This ends the project startup section, please add you thoughts and make corrections/clarifications.

--- Warning, proceeding by reading below this red line may cause confusion and/or deep thoughts. Serious project stuff goes above this line. ----

The notes/ideas below are from an old partly academic discussion regarding potential student projects, master thesis and PhD candidate work in this area. Some of the links may be dead by now. Feel free to add to the creative wiki-chaos here! More urgent and project related (less chaotic) stuff should go above the red line...

Use case background for online collaboration

[2014-note: online collaboration is not a priority in the initial phase above]

The current archetype editors are downloadable off-line applications and collaboration is done by editing offline, then sharing/uploading and discussing online (e.g. using products like Ocean Informatics CKM).

The situation above works and can be compared with collaborative editing of word processor documents. Document writing can be done in off-line applications and then shared and discussed online, or in mail sent back and forth. Versioning and conflict resolution of parallel edits (merging etc) requires skill, time and patience. Compared to this, it is often a lot easier to collaborate using online realtime multi-user editor environments like https://drive.google.com/, https://writer.zoho.com/, and http://en.wikipedia.org/wiki/Etherpad 

Many of the realtime environmants include some kind of comment/discussion mechanisms (e.g. the google drive apps) and history/playback/revisioning mechanisms that also sho who did what when. (Try “Time slider” at the etherpad-based http://piratepad.net/vDW7YbIMKv document and feel free to edit/experiment).

Just creating yet another archetype editor would not be such an interesting contribution to the world, but creating a new online collaborative archetype editor that has comments and timeline/playback functions would be a nice contribution!

When Google Wave (now Apache Wave) was active, the openEHR community tried different uses for it. Follow the threads around the message http://www.openehr.org/mailarchives/openehr-clinical/msg01615.html, and read http://omowizard.wordpress.com/2009/11/17/google-wave-and-health/ (Maybe also read some search results: https://www.google.se/search?q=openehr+google+wave) These links indicate that the real-time collaboration was appreciated for capturing knowledge in pre-editing phase of archetype creation.

Google wave was extendable with collaborative gadgets/widgets for mindmapping voting etc. There were open APIs that allowed you to add your own widgets (and server side “robots”). By combining client side gadgets/widgets with server-side robots, fairly sophisticated applications could be created.

At IMT, LiU Daniel Karlsson and Erik Sundvall supervised a student project that experimented with taking the archetyping discussions in Wave a bit closer to real archetype editing by creating widgets and robots that handled real partial archetype structures. The parts that got implemented in the limited timespan worked as expected. The editing process was automatically saved in the wave and could be played back using the wave playback function. Comments and discussion threads could be mixed in between the archetype editing widgets. During the project Google announced that it would shut down Wave, and the student project prototype was not continued by any follow-up projects.

Technical background

Many real-time multi-user online collaboration tools like the ones above are based on “operational transformation” (OT). A pedagogical description of OT is available at http://www.codecommit.com/blog/java/understanding-and-applying-operational-transformation

Some interesting OT-implementation links and discussions:

• A nice OT implementation is http://sharejs.org/ 

• ...others are discussed in e.g. http://stackoverflow.com/questions/2043165/operational-transformation-library 

• In that discussion e.g. https://code.google.com/p/google-mobwrite/  looks interesting

Operational transformation can be used on text as described above, but also data structurs like on XML (as was the case in Google Wave) and JSON (e.g. as described at https://github.com/josephg/ShareJS/wiki/JSON-Operations).

The openEHR AM can of course be represented as XML or JSON. The java-ref-impl provides XML import/export but JSON could be added (Erik S started experimenting with http://jackson.codehaus.org/ to provide AM/RM conversions to JSON)

Possible future

A master thesis project or similar effort create (or explore how to create) a real-time collaborative  online archetype editor. Ideally it should be possible to run the editor both in independent/freestanding mode and embedded in other collaborative environments (e.g. Google Hangouts https://developers.google.com/+/hangouts/ or similar platforms).

The broad idea is to host both pre-archetyping discussions and actual archetyping in the same (OT-based collaborative) environment that would take over most of what is done in off-line archetype editors today, a lot of what is done in the pre-editing phase, and some steps of what is done in the CKM today.

Some main ideas that can be reused from the LiU archetype-wave project are.

• Use some existing framework for real-time collaboration.

• Allow the archetyping discussion to start as a document with normal text, including widgets for bullet lists, mindmapping etc. Facilitate some way to include comments and discussion threads inside or connected to the document.

• Add widgets for editing parts (branches/leaves) of archetypes - think of it as breaking down existing archetype editors into smaller pieces. Allow these widgets to be placed anywhere in the document/discussion/wave. [Technology under the hood: Give the widget instance a UUID or similar and make sure that the widget updates its state using the features available in the real-time collaboration platform.]

• Add a tree-like widget for collecting and connecting the branch/leaf widgets into an archetype tree (without removing them from their original place in the document).

• Add server side functions for validating that the created archetypes or archetype parts follow the AM, RM and parent archetypes

• Use capabilities for named/numbered manually selected “revision points” (tags?) available in some real-time collaboration frameworks.

• A single “document” could contain several archetypes (by including several “tree-like” connector widgets).

Going further

• Import existing archetype(s) (e.g. to start review discussions)

• The document editing and discussion history (not only the finished result) should be possible to archive as an external file, useful for later playback.

• openEHR template editing [2014-note: With ADL 2.0 this should be included from start /Erik]

• make it ‘social’ (a la google+ hangouts)

Going even further (likely something for later additional projects after an initial online-editor-focused student project)

• Offline capacity in browser based applications is possible in modern browsers, see e.g. http://www.html5rocks.com/en/features/offline This could be used to have (all or some selected) archetypes and discussions available (read-only) when offline (e.g. when travelling). If implemented cleverly it will use HTML5 features for updating and downloading changes when reconnected.

• HTML5 offline features combined with a local instance of the server-side archetype-validation features (e.g. a small java-based server app) could be used to reduce or eliminate online server calls when editing archetypes alone. Collaborating with others that do parallel edits to the same archetype will of course re-introduce the same merging and conflict resolution problems as the off-line approach of today has.