Heather Leslie^a,b, Silje Ljosland Bakke^a,c

^a Co-lead, Clinical Modelling Program, openEHR Foundation,

^b CMIO, Ocean Health Systems, Australia

^c Information Architect, Nasjonal IKT HF, Norway

Abstract

Over the past 8 years the openEHR Clinical Model program has been developing a Web 2.0 approach and tooling to support the  development, review and governance of atomic clincial information models, known as archetypes. This paper describes the background and review process, and provides a practical example where cross standards organisation collaboration resulted in jointly agreed clinical content which was subsequently represented in different implementation formalisms that were effectively semantically aligned. The discussion and conclusions highlight some of the socio-technical benefits and challenges facing organisations who seek to govern atomic clinical information models in a global and collaborative online community.

Keywords:

Informatics, crowdsourcing, common data elements

Introduction

Historically, most collaboration in the health technology domain has been through formal balloting of message or document specification standards within standards development organisations (SDOs) such as ISO TC215 or HL7 International. This approval process has had some significant success over many years in supporting interoperability of health data, however this approach is not transparent, responsive or agile enough for development, maintenance and governance of larger numbers of more atomic clinical information models.

The Clinical Models Program [1] at the openEHR Foundation [2] has developed an alternative, crowdsourced approach to the development, publication and governance of the openEHR clinical information models, known as archetypes. This methodology emphasises openness, transparency and accountability to the community.

The Clinical Knowledge Manager tool was developed directly as a result of experience of the openEHR Clinical Program leads in working with distributed groups using technical tools such as widely used software versioning and revision control systems. It quickly became apparent that there was not only a need for versioning governance but also life cycle and naming governance plus a critical process to ensure appropriate review of the archetypes to ensure that each was fit for use in implementations.

This openEHR methodology is now in use by a number of national and jurisdictional eHealth programs around the world who are using archetypes published in this manner to underpin their local health IT infostructure.

Background

The Clinical Models Program at the openEHR Foundation is responsible for management of a set of clinical information models, known as archetypes, on behalf of the international openEHR community. The scope of responsibility includes:

• development of a set of coherent and consistent archetypes,
• commumity review and approval of the archetypes as fit for use, using a collaborative peer-review process;
• publication and life cycle management of each archetype; and
• ongoing maintenance and governance of each archetype.

Each archetype is a computable specification for a single clinical concept – intended to be a maximal data set for a universal use case. In practice, the aspirational intent of a maximal data set is usually adjusted to a practical, but inclusive, data set that can be re-used across multiple clinical scenarios.

The program utilises an online tool to support the activites of the program – the openEHR Clinical Knowledge Manager (CKM) [3]. This tool has three main purposes – it is a public library of archetypes; an open collaboration portal; and underpins the community’s requirements for complex clinical knowledge governance.

The CKM tool allows open access to all clinical information models and associated supporting information. In order to actively participate with the openEHR commuity within the CKM tool, registration is required. Registration within the CKM portal is free and open to any interested individual or group to maximise participation from a broad range of professions including, but not limited to:

• Clinicians;
• Informaticians;
• Software engineers;
• Terminologists;
• Academics/students;
• Administrators; and
• Consumers.

As of December 18, 2016, the openEHR CKM has:

• 500 active archetypes in varying life cycle stages, comprising an estimated 6000 data points; 
• 1628 registered users from 88 countries: and
• 24 languages represented.

There are other communities using separate CKM instances in Norway, Australia, United Kingdom, Slovenia, Canada and Brazil. These groups actively share archetypes and collaborate together to minimise ‘reinventing the wheel’ and [BSL3] 

openEHR peer-review process

A small number of Clinical Knowledge Administrators are appointed to take responsibility for the operations of CKM as a a whole. They appoint Editors who are charged with  the responsibility to develop and enhance the clinical content of each archetype from its initial draft through to a published life cycle state. The CKM tool support this iteration by enabling the Editors to run a series of review rounds to gather and collate reviewer feedback and manage the associated version and audit controls.

An archetype review round is manually initiated by an Editor. They invite a subset of registered CKM reviewers - selected to ensure an appropriate cross section of professions, health domains, geographical location are represented. In addition, anyone who has a special interest in participating and requests an invitation to that review will also be included.

Review rounds are typically sent out for a period of two weeks. This is not fixed and can be adjusted by Editors on a per review round basis. Reviewing is optional and reviewers can choose whether or not they participate in each review round for each archetype.

Each review comprises a ‘wizard’ process that supports the reviewer to be able to make comments on various components of the archetype and respond to a standard series of questions. The only mandatory response is a final recommendation about the state of the clinical content of archetype:

• Accept – ready for publication without further community review;
• Minor Revision – only trivial changes (usually spelling/grammar) are required, otherwise ready for publication without further community review;
• Major Revision – significant changes are needed, requiring further community review;
• Reject – not fit for publication or fundamentally flawed: and
• Abstain – no recommendation.

At the end of the review period the Editors gather together, usually via teleconference, to respond to the feedback, make changes to the archetype and decide on the next steps for the archetype. This depends on the recommendations submitted by reviewers – if all recommendations are ‘Accept’ or ‘Minor Revision’ then consensus has been achieved for the archetype to be published; if ‘Major Revision’ or ‘Rejected’ are recorded, then further review rounds are usually required until consensus is achieved.

All review comments plus the responses of the Editors to each reviewer comment is captured and made available to any other registered use. In this way, the process is transparent and the Editors can be held accountable to the community.

Approach

The following narrative outlines the openEHR approach by describing a complex collaboration between the openEHR and HL7 communities, using the recent experience of cross SDO review of clinical content for representation of Adverse Reaction Risk, also referred to as Allergy/Intolerance within the HL7 community. The narrative has been pieced together retrospectively from audit train and review date recorded in the respective CKM tools.

The very first iteration of the draft candidate for the Adverse Reaction archetype was authored in April 2006 by a single Australian clinical informatician, Dr Sam Heard. It was one of the first archetypes uploaded to the openEHR CKM [3] on 23 July 2008 [4]. On 3 July 2009 this archetype commenced its’ first collaborative peer-review in the openEHR CKM.

On 9 November 2010, Australia’s National eHealth Transition Authority (NEHTA, now known as the Australian Digital Health Agency [5]) forked the archetype and brought it into the Australian CKM [6,7] environment and ran a series of five archetype reviews during the period through to June 2011. The resulting archetype formed the basis for the adverse reaction data elements in the initial PCEHR CDA documents which are used to transmit health data from Australian primary care clinical systems into the PCEHR (now known as ‘My Health Record [8]’).

The next major iteration of the international archetype by Dr Heather Leslie included major changes based on feedback from reviewers plus input from concurrent Australian CKM reviews, and a variety of other resources including academic papers [9,10] and documents published and available at the time by NHS England [11,12], Microsoft’s Clinical User Interface group [13,14], and the Royal Australian College of General Practitioners [15].

It was uploaded as a fork to the international CKM on January 16, 2012 [16] and on the same day another peer-review round was commenced.

In May and June 2014, further work was completed by Dr Ian McNicoll that blended the feedback from the 2012 review, and content from HL7’s FHIR resource at the time and RMIM publications. On 3 June 2014 it was uploaded as a new archetype [17] – the ‘Adverse Reaction (FHIR/openEHR)’ archetype with the intent of conducting a series of joint FHIR and openEHR reviews on the combined archetype and generating both a FHIR resource AND an openEHR archetype with matching, clinically verified content at the end of the process.

On 7 July 2014 the first joint openEHR/FHIR review was initiated. Four expert Editors were appointed to facilitate the reviewer feedback – two acting on behalf of openEHR and two from HL7. The resulting archetype was uploaded into the international CKM on 22 October 2014 and a second review round initiated.

Concurrently, the Norwegian Nasjonal IKT team forked the archetype into the Norwegian CKM [18] on 26 November 2014.

Resolution of the second openEHR/FHIR review round did not occur until nearly 7 months later, in June 2015, due to the need to wait for the FHIR community to complete a full FHIR ballot and incorporate this feedback into the next archetype version, which was uploaded to the international CKM on 11 June 2015 [19].

On 11 June 2015 the third openEHR/FHIR review round commenced in the international CKM. Concurrently in the Norwegian CKM, the archetype was updated to be aligned with the international archetype and it was translated to Norwegian [20]. Subsequently on 2 August 2015 the first review round in Norwegian commenced in the Norwegian CKM. Feedback from this review was added to the international feedback so that the archetype development could evolve with English and Norwegian versions in parallel and now collectively incorporating feedback from each of the international openEHR, Norwegian and HL7 communities. The resulting archetype was uploaded to the international openEHR CKM on 22 October 2015 [21] and on the same day the fourth and final openEHR/FHIR review round commenced in the international CKM. On 29 November 2015 the second parallel review round commenced in the Norwegian CKM using the latest aligned and translated version of the archetype [22].

At the completion of the November 2015 review round and analysis of reviewer feedback, the editors agreed that a consensus had been reached amongst the participants. In the openEHR archetype, all FHIR-specific components were removed and published the ‘Adverse reaction risk’ archetype was published. The status of the original archetype was simultaneously changed to rejected – this rejected archetype persists in the international CKM as part of the provenance/audit trail for the published archetype but clearly marked as non-current.

The Norwegian CKM updated their version of the archetype to align the content and the translation. They initiated a final review in Norwegian, commencing 29 Nov 2015. At its conclusion the Norwegian archetype was also published within their local CKM and is now governed autonomously by the Nasjonal IKT team as per their national mandate. The agreed intent of the international and Norwegian teams is to continue to collaborate when change requests arise or new requirements are identified. The two archetypes remain semantically aligned as of December 18, 2016.

An independent FHIR resource evolved throughout the CKM review process, adopting the changes agreed through the review process. This was the artefact that was reviewed by the FHIR community. At the time of archetype publication, the content of the archetypes and the FHIR resource were aligned.

Results

It has been possible to collate review related data from each of the three CKM instances that have been used as part of the evolution of the Adverse Reaction Risk archetype through to publication.

A diagrammatic view of the review process is shown in Figure 1.

The statistics about the contributors for each CKM review round is shown below in Table 1. The table identifies that across all three CKM instances:

• 13 review rounds were completed;
• 126 contributors participated, mostly from individuals but a small number were received from a group who submitted collated review comments; and
• 203 contributions were received as part of the review process.

Table 1– Contributors statistics for each archetype

Unfortunately the parallel processes in the HL7 ballot processes are not available to include in this analysis. So the actual numbers of contributors will be much larger than indicated by these numbers, but we have no indication of the size of this contribution.

Discussion

This Adverse Reaction Risk archetype started its’ journey as the brainchild of a single clinical informatician. It was published with the collective input of over 126 contributors, each contributing according to their professional background and expertise.

There has been no further formal joint collaboration between the openEHR and FHIR communities since the November 2015 publication. At that point in time, the great majority of content in both the openEHR archetype and the FHIR resource were aligned as a consequence of the joint review process.

Figure 1 – Cross SDO Adverse Reaction Risk archetype collaboration process

Subsequently it appears that the FHIR resource has continued to evolve in isolation, effectively diverging from the jointly agreed artefact, due to further requirements being identified in HL7 implementations [23]. This is unfortunate and not the desired outcome that was hoped for by the openEHR Clinical Program Leads, however it does highlight that for successful and ongoing coordination and collaboration on standardisation and alignment of clinical information models, all participants need to keep this as a priority.

However, the ongoing collaboration between the openEHR CKM and Norwegian CKM teams is active and ongoing, and is successful largely because both groups are committed to working together and sharing the Editorial work required to facilitate the reviewer feedback.

Inclusion of the HL7 community was valuable in order to gather broader expert input and has no doubt improved the quality of the archetype. The final archetype was agreed in terms of the clinical content and then a pure openEHR archetype and a corresponding FHIR resource were developed, based on that common clinical content. This was a significant achievement for both communities, however the subsequent isolated divergence of the HL7 clinical content is disappointing but likely inevitable without a strong commitment to maintaining alignment. From the openEHR point of view, another drawback is that by trying to fit within the HL7/FHIR balloting process, the timelines to publication was extended from an expected three to six months for a complex information model to actually take eighteen months.

Interestingly, one HL7 member responded to an invitation to a two week archetype review round by declining participation on the basis that it would be impossible for them to respond to the archetype review in less than 6 months – this for a single information model containing twenty one data points plus metadata. In the experience of openEHR Editors it would be expected that the time required for this review would typically be be approximately thirty to sixty minutes total elapsed time.

The traditional SDO process is usually a closed activity in which value is placed on participation only by credentialled individuals, determined either by financial membership or nomination as an expert.

By contrast, in the global Web 2.0 crowdsourced environment in which the openEHR communities of interest operate, the opposite conditions largely apply. The openEHR methodology places enormous weight on broad participation, accountability of those in roles of authority to every member of the community, and transparency at every level of governance:

1. Participation is open and free – participation is open to anyone who is willing to participate to the extent of their ability. It is not limited to individuals or organisations who have current paid memberships, who are nominated as ‘experts’, or who have been designated as ‘credentialled’ experts. This may be challenging to many but it supports input from the broadest professions, health domain expertise and geographical sources.
2. Everyone can participate according to their expertise. The user interface and review processes in the CKM tool has been developed specifically to ensure that non-technical experts, such as grassroots clinicians, can participate equally alongside the technology savvy. It removes the need for clinicians to acquire additional technical skills in order to participate. All feedback is encouraged, from the smallest grammatical correction through to those who complex informatics or implementation solutions.[BSL6]
3. Transparency. All of the activities and decision-making withing CKM is transparent to registered users, including but not limited to:
   • • Archetype reviews, especially:

-   acknowledgement of all participants and their roles;

-   clear association between reviewer comments and resulting Editorial decisions;

-   number of contributions; number of review rounds; and

-   professional and health domain expertise background of all reviewers to ensure that an appropriate reviewer community has been involved.

• • Threaded, unmoderated discussion threads;
  • Change requests by registered users and Editorial responses; and
  • Archetype audit trail.

If a registered user is not happy with decisions there are a number of ways of raising this with Editors or via public discussion boards.

4. Rapid and agile archetype publication. In the work we have done to date, the typical archetype review process involves 4 review rounds to achieve broad agreement on the structure and data points. Sometimes further review rounds are required, usually focussed on refinement of archetype descriptions and metadata. With an average review round duration of two weeks, this means that an archetype requiring six review rounds could potentially be published in twelve weeks. Archetypes based on established and agreed clinical content such as evidence-based scales and scores can often be published in one or two review rounds – corresponding to between two and four weeks.

Assuming modest Editorial resources are available, when multiple archetypes are being reviewed simultaneously it is possible to publish archetypes in an efficient and effective timeframes.

By contrast, the traditional SDO ballot process would not be sustainable in the openEHR environment where the intent is to develop, review and publish all clinical archetypes required for all clinical data recording. There is a practical need for archetype review rounds to be:

• • Managed as a sequence of short, frequent review rounds that result in progressively refined iterations of the archetype;
  • Initiated independently of other archetypes and for a variety of reasons, including initial publication, management of change requests and maintenance processes; and
  • Run when required - sometimes in parallel with other archetype reviews and at other times on an ad hoc basis to resolve a specific issue.

5. Shared archetypes amongst communities. There are now a number of groups using the CKM tool as the basis of national or jurisdictional standardisation of data sets.

The traditional SDO process does not usually reveal the primary authors or contributors to their published standards, although they will possibly be known to SDO members. However the openEHR approach places enormous weight on transparency at every level of governance and for Editors to be accountable to the CKM community:

• • Free and open membership;
  • Detailed audit trails to ensure accurate provenance and recording of Editorial changes;
  • Visibility of reviewer contributions and Editorial responses
  • Statistics about the review process, including:

-   acknowledgement of all participants;

-   number of contributions;

-   number of review rounds; and

-   background of all reviewers to ensure an appropriate reviewer community.

Conclusion

Clinical information modelling governance has been a new and largely untested challenge until recently – most of our experience in governance of health data standards has been at the complete message or document data set level. The Clinical Knowledge Manager tool was developed directly in response to identification of the need for efficient and responsive iterative refinement of the archetypes in response to identified requirements, especially during implementations – finding the sweet spot in the tensions between governance and evolution to ensure that the information models were safe and fit for use.

Clinical knowledge governance is a complex, evolving and poorly understood domain. Technical governance is one critical aspect but the main challenges were actually socio-technical, related to clinciain engagement and participation, models for collaboration and governance of a set of artefacts that needed to dynamically evolve with a multitude of dependencies governance during implementations.

Disclosure

The Clinical Knowledge Manager tool was developed by Dr Heather Leslie and Ocean Health Systems.

References

[1]    Clinical Models Program [Internet]. London: openEHR Foundation; [cited: 2016-12-23]. Available from: http://www.openehr.org/programs/clinicalmodels/

[2]    openEHR Foundation Home [Internet]. London: openEHR Foundation; [cited: 2016-12-23]. Available from: http://www.openehr.org/

[3]    openEHR Clinical Knowledge Manager [Internet]. London: openEHR Foundation; [cited: 2016-12-23]. Available from: http://www.openehr.org/ckm/

[4]    Adverse Reaction, Rejected Archetype [Internet]. openEHR Foundation, openEHR Clinical Knowledge Manager [cited: 2016-12-23]. Available from: http://www.openehr.org/ckm/#showArchetype_1013.1.197_1

[5]    Australian Digital Health Agency Home [Internet]. Sydney, Australia: Australian Digital Health Agency; [cited: 2016-12-23]. Available from: http://www.digitalhealth.gov.au/

[6]    Australian Digital Health Agency Clinical Knowledge Manager [Internet]. Sydney, Australia: Australian Digital Health Agency; [cited: 2016-12-23]. Available from: http://dcm.nehta.org.au/ckm/

[7]    Adverse Reaction, draft archetype, National eHealth Transition Authority [Internet]. NEHTA Clinical Knowledge Manager. Authored: 08 Nov 2010. Available at: http://dcm.nehta.org.au/ckm/OKM.html#showarchetype_1013.1.868_7 (accessed Jan 16, 2012).

[8]    My Health Record Home [Internet]. Sydney, Australia: Australian Digital Health Agency; [cited: 2016-12-23]. Available from: https://myhealthrecord.gov.au/internet/mhr/publishing.nsf/content/home

[9]    Riedl MA, Casillas AM. Adverse drug reactions: types and treatment options. Am Fam Physician. 2003 Nov 1;68(9):1781-90. Review. PubMed PMID: 14620598.

[10] Thien FC. Drug hypersensitivity. Med J Aust. 2006 Sep 18;185(6):333-8. Review. PubMed PMID: 16999678.

[11] Horsfield P, Sibeko S. Representation in Electronic Patient Records of Allergic Reactions, Adverse Reactions, and Intolerance of Pharmaceutical Products [Internet]. London, UK: National Health Service; 2006 Sep 07 [cited: 2016-12-23]; Available at http://webarchive.nationalarchives.gov.uk/+/http://www.isb.nhs.uk/documents/isb-1582/amd-24-2011/1582242011npfitepdb.pdf.

[12] Long R, Bentley S. SCG Guidance on the Representation of Allergies and Adverse Reaction Information Using NHS Message Templates [Internet]. London, UK: National Health Service; 2008 Apr 30 [cited 2011 Jun 21]; No longer available at http://www.connectingforhealth.nhs.uk/systemsandservices/data/scg/scg0001.pdf.

[13] Microsoft. Design Guidance: Displaying Adverse Drug Reaction Risks [Internet]. 2009 January 28 [cited: 2016-12-23]; Available at http://www.mscui.net/DesignGuide/DisplayingAllergies.aspx.

[14] Microsoft. Design Guidance: Recording Adverse Drug Reaction Risks [Internet]. 2009 March 27 [cited: 2016-12-23]; Available at http://www.mscui.net/DesignGuide/Pdfs/Design%20Guidance%20--%20Recording%20Adverse%20Drug%20Reaction%20Risks.pdf.

[15] Royal Australian College of General Practitioners. Fact Sheet: Allergies & Adverse Reactions (Draft). 2010.

[16] Adverse Reaction, Rejected Archetype [Internet]. openEHR Foundation, openEHR Clinical Knowledge Manager [cited: 2016-12-23]. Available from: http://www.openehr.org/ckm/#showArchetype_1013.1.197_5

[17] Adverse reaction risk, Published Archetype [Internet]. openEHR Foundation, openEHR Clinical Knowledge Manager [cited: 2016-12-23]. Available from: http://www.openehr.org/ckm/#showArchetype_1013.1.1713_1

[18] Nasjonal IKT Clinical Knowledge Manager [Internet]. Oslo, Norway: Nasjonal IKT HF; [cited: 2016-12-23]. Available from: http://arketyper.no/ckm/

[19] Adverse reaction risk, Published Archetype [Internet]. openEHR Foundation, openEHR Clinical Knowledge Manager [cited: 2016-12-23]. Available from:http://www.openehr.org/ckm/#showArchetype_1013.1.1713_15

[20] Risiko for overfølsomhetsreaksjon, Published archetype [Internet]. Nasjonal IKT, Nasjonal IKT Clinical Knowledge Manager [cited: 2016-12-23]. Available from http://arketyper.no/ckm/#showArchetype_1078.36.579_2

[21] Adverse reaction risk, Published Archetype [Internet]. openEHR Foundation, openEHR Clinical Knowledge Manager [cited: 2016-12-23]. Available from: Available from: http://www.openehr.org/ckm/#showArchetype_1013.1.1713_16

[22] Risiko for overfølsomhetsreaksjon, Published archetype [Internet]. Nasjonal IKT, Nasjonal IKT Clinical Knowledge Manager [cited: 2016-12-23]. Available from: http://arketyper.no/ckm/#showArchetype_1078.36.579_5

[23] Graham Grieve. FHIR and openEHR [Internet]. HL7 International; 2016 Nov [cited: 2016-12-23]. Available from: https://vimeopro.com/user12740828/hl7-fhir-developer-days-2016-amsterdam/video/191939835

Address [HL7] for correspondence

Dr Heather Leslie - heather.leslie@oceanhealthsystems.com