openEHR Conformance

Conformance testing is used as the basis of product certification, and has the following goals:

Tendering: enable tendering authorities to state formal criteria for compliance in tenders
Protection for solution developers: enables bona fide vendors and other developers to prove the quality of their solutions compared to other offerings claiming conformance
Protection for procurement: provides a way of ensuring that purchased solutions can be contractually guaranteed to perform in certain ways

Conformance is tested in two categories:

Functional: test correctness of implementation with respect to specifications, represented by specific test sets
Performance: test capacity of system to process data and transactions

All conformance specifications, test sets and related materials are part of the Conformance Specification component.

References:

Conformance test design

Conformance testing on a product for a given test category is is performed by:

select a specific test set for that category
setting up initial conditions, e.g. inject a template
executing specific transations on a given API or other interface (e.g. query)
retrieving result
performing regression comparison of actual v reference

Conformance schedule - Functional

Conformance level	Test Category	Description	Inputs	Outputs & Conformance Points	Test sets	Capabilities exercised
1	Template injection test	End-to-end template => data test. Inject a new template Instantiate from screen form or TDD Export canonical XML	Composition Template	Regression test exported openEHR canonical XML v test reference XML	TBD: Ian's monster template	Archetypes Templates RM Canonical XML
1	EHR API lifecycle test	EHR create, commit, read via EHR API. create new EHR Contribution 1: admin Composition (admission) Contribution 2: persistent Composition (meds) Contribution 3: event Composition (vital signs) Contribution 4: update meds + event Composition (vital signs) display & output all Contributions & meta-data as JSON display & output latest version of Compositions in canonical XML display & output all versions of Compositions in canonical XML	Composition Templates	?Contribution JSON v regression JSON Regression test exported openEHR canonical XML v test reference XML	Template set: admission medications vital signs	EHR API Contributions and versioning
2	TDD round trip	Data commit & retrieve using TDD based on provided TDS instantiate TDD from TDS by any method commit TDD export as canonical XML	TDS	Regression test exported openEHR canonical XML v test reference XML	Vital signs template
2	Basic AQL	Test AQL query against single data set (template). Commit vital signs OBS x 50 samples Execute query and return result set XML	Template Test query	Regression test Result set XML v reference XML	Vital signs OBS template: BP heart rate body temp SaO2 VS Test query	Basic AQL AQL result set
3	REST API
3	Advanced AQL	Exercise AQL queries with WHERE clause across multiple data sets (templates)	Templates Test query	Regression test Result set XML v reference XML		AQL WHERE processing
OPTIONAL	Demographics API lifecycle test	Exercise openEHR demographic content create and retrieve
OPTIONAL	EHR Extract export

Conformance schedule - Performance

Conformance
level

Test Category

Description

Inputs

Outputs & Conformance Points

Test sets

Capabilities exercised

1

Basic

Minimal level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate

2

Medium

Medium level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate

3

High

High level of:

Concurrent users / screen latency
Commit transaction rate
Query transaction rate