openEHR Conformance

Overview

Conformance testing is used as the basis of product certification, and has the following goals:

Tendering: enable tendering authorities to state formal criteria for compliance in tenders
Protection for solution developers: enables bona fide vendors and other developers to prove the quality of their solutions compared to other offerings claiming conformance
Protection for procurement: provides a way of ensuring that purchased solutions can be contractually guaranteed to perform in certain ways

Conformance is tested in two categories:

Functional: test correctness of implementation with respect to specifications, represented by specific test sets
Performance: test capacity of system to process data and transactions

All conformance specifications, test sets and related materials are part of the Conformance Specification component.

References

Conformance test design

Conformance testing on a product for a given test category is is performed by:

select a specific test set for that category
setting up initial conditions, e.g. inject a template
executing specific transations on a given API or other interface (e.g. query)
retrieving result
performing regression comparison of actual v reference

Conformance schedule - Functional

Conformance level	Test Category	Description	Inputs	Outputs & Conformance Points	Test sets	Capabilities exercised
1	Template injection test	End-to-end template => data test. Inject a new template Instantiate from screen form or TDD Export canonical XML	Composition Template	Regression test exported openEHR canonical XML v test reference XML	TBD: Ian's monster template	Archetypes Templates RM Canonical XML
1	EHR API lifecycle test	EHR create, commit, read via EHR API. create new EHR Contribution 1: admin Composition (admission) Contribution 2: persistent Composition (meds) Contribution 3: event Composition (vital signs) Contribution 4: update meds + event Composition (vital signs) display & output all Contributions & meta-data as JSON display & output latest version of Compositions in canonical XML display & output all versions of Compositions in canonical XML	Composition Templates	?Contribution JSON v regression JSON Regression test exported openEHR canonical XML v test reference XML	Template set: admission medications vital signs	EHR API Contributions and versioning
2	TDD round trip	Data commit & retrieve using TDD based on provided TDS instantiate TDD from TDS by any method commit TDD export as canonical XML	TDS	Regression test exported openEHR canonical XML v test reference XML	Vital signs template
2	Basic AQL	Test AQL query against single data set (template). Commit vital signs OBS x 50 samples Execute query and return result set XML	Template Test query	Regression test Result set XML v reference XML	Vital signs OBS template: BP heart rate body temp SaO2 VS Test query	Basic AQL AQL result set
3	REST API
3	Advanced AQL	Exercise AQL queries with WHERE clause across multiple data sets (templates)	Templates Test query	Regression test Result set XML v reference XML		AQL WHERE processing
OPTIONAL	Demographics API lifecycle test	Exercise openEHR demographic content create and retrieve
OPTIONAL	EHR Extract export

Conformance schedule - Performance

NB: There may be better schemes for formalising performance levels - please comment.

Conformance
level

Test Category

Description

Inputs

Outputs & Conformance Points

Test sets

Capabilities exercised

1

Basic

Minimal level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate

2

Medium

Medium level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate

3

High

High level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate