CKM-444

Reusable Review Invitation Text Templates

When editors initiate a review round, they need to supply an invitation text that is displayed to all reviewers on starting their review.

To better support editors in initiating review rounds, functionality is added to CKM to define and then use and reuse review invitation templates.

For this, CKAs can define review invitation templates with the following characteristics:

• Each review invitation template has a unique name

• One set of review invitation templates is managed in each CKM instance, available to all editors

• Each Invitation template can either be for the invitation text displayed in CKM or the (new) mail invitation text, or both.

This functionality is available from the Reviews/Manage Invitation Templates menu.

When creating a new review round, editors can select any applicable preconfigured review invitation template. This is done via a Button to import the invitation template which opens a window to select the desired invitation template. The selected review invitation template is then displayed in the usual review invitation text RichTextArea in CKM and can be finetuned for use in this particular review round, using the existing editing functionality.

Any Review Invitation Text Template can contain a selection of placeholders to better enable the reuse. The following placeholders can be used in the Review Invitation Template:

• Reviewer:

  • %{reviewer-first-name}%

  • %{reviewer-last-name}%

  • %{reviewer-full-name}%

  • %{reviewer-login-name}%

• Editor(s):

  • %{editor-full-name}%

• Resource:

  • %{resource-type}% - archetype, template, …

  • %{resource-type-definite}% - The definite form: In Norwegian, e.g. “arketypen” instead of “arketype”.

  • %{resource-main-display-name}% - the main display name of the resource (e.g. concept name of an archetype) in the best available language (based on the CKM server’s primary language configuration)

  • %{resource-main-id}% - The main id, e.g. the archetype id.

• Review Round:

  • %{review-type}% - content, translation, terminology

  • %{translation-language}% - The translation language for translation reviews.

  • %{terminology}% - The terminology under review for terminology reviews.

Placeholders need to be wrapped like this: %{ … }% - for example: %{resource-type}%

All placeholders are wrapped in percentage signs and curly brackets. To avoid problems with these placeholders, the invitation template cannot be stored if it contains an unsupported placeholder. Neither can a review round be created nor can reviewers be invited if non supported placeholders are found in any invitation text. 

Note that there are two types of invitation templates. Inline invitations are displayed inline in CKM when a reviewer starts to conduct a review. Custom mail invitation texts are ADDED to the usual invitation email in order to customise these emails, see CKM-1421: Ability to add additional (e.g. motivating) text to the review invitation email itself for details.

Example invitation:

Dear %{reviewer-first-name}% %{reviewer-last-name}%, 

You are invited to review %{resource-type}% %{resource-main-display-name}% with %{resource-main-id}%. 

In some languages, the definite form of a noun differs from the indefinite form and can be used like this for resource type: %{resource-type-definite}%. 

The review type is %{review-type}%. 

For a translation review, the translation language to be reviewed is: %{translation-language}%. 

For a terminology review, the terminology under review is %{terminology}%. 

These will be empty if not applicable. 

The reviewer can also be addressed with the reviewer's full name: %{reviewer-full-name}%. It is also possible to provide the reviewer's login name: %{reviewer-login-name}%. 

Kind regards, the editor(s): %{editor-full-name}% 

CKM-1274

Improve warnings displayed before committing or publishing a resource (or initiating a review round) to look more like a checklist

When a resource is being committed or published, any warnings to check the archetype before committing or publishing are currently displayed like this:

Open

Likewise, these warnings are displayed when initiating a review round, however the layout is horizontal to save some space in the review round creation wizard.

The display of these warnings has been improved to be be more like a checklist that shows what has been validated and where there are still warnings or errors. The number of errors/warnings has been added as a badge, and if there aren’t any errors/warning of a given type, then a green badge with a tick is used to clearly state that there no errors/warnings of the type, for example:

CKM-1421

Ability to add additional (e.g. motivating) text to the review invitation email itself

With this issue, we add the ability for editors to define an individual text that is included in the review invitation email. This email is sent out to the reviewers when an editor initiates a new review round.

This text would usually be different to the complete review invitation text that is displayed to reviewers when starting their review in CKM. The latter is usually a fairly lengthy text with detailed instructions on how to review on what to take into account.

The reason for adding the ability to specify an additional text for the email invitation (“mail-invitation-details”) is to increase the likelihood that a potential reviewer is interested in reviewing the resource under review in the first place and accepts the review invitation. This separate mail-invitation-details field that can be filled in during review round setup via a new step. This text then added to current invitation email sent out to the reviewers, just before the “Your Options:” paragraph.

This functionality has been designed for editors to have maximum flexibility: Editors can provide completely different or also exactly the same content for both the invitation details displayed to reviewers in CKM and in the email. Of course, they can also completely omit the additional email invitation text, which simply preserves the status quo of the current review invitation emails. In case that the content of the two invitation details (email vs. in CKM) overlaps, the editors would need to enter (copy-paste) the relevant parts at both places when initiating the review round. To simply this, CKM-444: Reusable Review Invitation Text Templates also introduces reusable invitation text templates.

CKM-1455

Warn the user about uploading a resource to an outdated branch BEFORE submitting

When a user uploads an archetype to a branch and that branch is outdated, for instance, because another branch has been committed to the trunk in the meantime, this may cause problems or simply unnecessary editorial work down the line.

While it is - and should be - possible to continue work on the branch and integrate the changes from the trunk accordingly, it may in some cases be preferable to start again with a new branch.

Therefore, CKM will now present an warning when an archetype or template is uploaded BEFORE it is actually submitted to the outdated branch. The user can then consider to reject/resolve/delete the outdated branch first. This warning is in addition to the warning the user currently already gets after the submission to the branch has been completed.

CKM-1475

Editor functionality to show the revision changes for each review feedback element

When multiple editors work collaboratively on the feedback for a review round, it has so far not been possible to see which editor contributed to the Editor feedback in the reviews, i.e. added/changed and maybe even deleted the feedback for an element.

This is of course desirable when displayed to the reviewers - the editors speaking with one voice.

However, when there are multiple editors, it is useful as an editor to see the editor and timestamp for each feedback element. This is to be able to reconstruct and streamline editorial discussions (especially afterwards).

As each item could be changed multiple times, the whole history of who changed what when is relevant for each element under review. For this purpose, a new button is added next to the button to “Add New Task” for each feedback element. This button (like the existing “Add new task” button) is only displayed when the feedback element is focussed and will display the complete Feedback History for the element.

CKM-1536

As an Editor, access the Reviewer's User Profile directly from Completed Reviews and Review Invitations

For editors, it should be possible to directly go to the user profile of a reviewer from the following grids:

• The “Completed Reviews” grid available for each review round.

• The “Review Invitations” grid. This grid can for example be accessed via right-click on a Review Round in the Review Rounds Overview and then selecting “Show invitations of this review round” from the context menu.

If an editor right-clicks on a user in these grids, the editor can now select the new “View User XYZ” menu item to directly access the user profile.

CKM-1539

Enhance the Archetype Data Points report

The Archetype Data Points Report has been enhanced in the following ways:

• Added the total number of archetypes evaluated.

• Added the total number of data points.

• Added the number of non-parsable archetypes if any archetypes could not been evaluated due to a parsing problem.

• Added the average number of data points per archetype.

• Added the percentage for each data type in grid (in addition to the total number).

CKM-1554

Bulk Create/Update Users via CSV File Upload

A Clinical Knowledge Administrator (CKA) or Technical Administrator - both referred to as admin in the following - bulk creates users by uploading a csv file containing specific user details. Under certain conditions, details of existing users can be updated using the same approach.

This use case starts when an admin is in possession of a CSV file with user details and wants to bulk create accounts for these users in CKM (or update some of the user details). This use case ends when CKM has created/updated the user accounts and provided respective feedback to the admin.

CSV File Structure

1. The csv file starts with the first row containing the correct fields in correct order: first-name, last-name, email-address, etc.

2. The csv files contains any number of subsequent rows with first name, last name, email address, etc.

Required columns:

• first-name: The user's first name.

• last-name: The user's last name.

• email-address: The (primary) email address of the user.

• country-code: The country code based on ISO 3166-1 alpha-2, e.g. CA or AU.

Optional columns:

• organisation: The name of the user’s organisation/affiliation

• health-domain: One health domain from CKM’s flexible ontology representing the user’s most relevant health domain.

• profession: One profession from CKM’s flexible ontology representing the user’s most relevant profession.

The username is constructed from <first-name>.<last-name> and does not need to be specified in addition.

The secondary (recovery) email address is not commonly stated as part of the registration process and new users will be asked if they want to provide one at a later stage within CKM as per normal CKM processes.

Health Domain and Profession

If health domain and/or profession are specified, the name of a specified health domain and profession needs to be provided exactly (case-sensitive) as in CKM, including the complete hierarchy, separated by forward slashes (“/”):

• For example for health domain: Otolaryngology/Ear, Nose & Throat

• For example for profession: Clinicians/Allied Health Professionals/Audiologist

Note: If a class in itself contains a forward slash “/” this MUST be replaced with a pipe symbol (“|”), for example for Health domain:
Emergency|Ambulance must be used instead of Emergency/Ambulance.

Additional Input Options

In addition to the input from the CSV file, it is possible to make some additional choices in the CKM GUI that apply to all users to be created:

1. A project or incubator can optionally be selected: new users are added to this project/incubator in a role (reviewer).
   This is an alternative way to support subsequently inviting users to review rounds.

2. Activate user account immediately:

   1. If yes, the users are created and a Welcome email is sent to each new user.

   2. If no, the users are created and an Activate Your Account email is sent to each new user.

3. Available as reviewer: Is the user available as a reviewer? This requires three states: Yes, No, User to decide.

4. Available as translator: This requires three states: Yes, No, User to decide.

5. A default password (of 6 or more characters) for all new users.

   1. If provided, the admin will notify the users of the default password separately.

   2. If no password is provided, CKM will generate a (different) initial password for each user.
      This password can be reset and retrieved by the user via CKM’s Forgot Password functionality.

Preconditions

1. The CKM user is signed in and has one of the following roles: Clinical Knowledge Administrator (CKA) or Technical Administrator - in the following referred to as admin for simplicity.

Basic Flow

1. The admin selects “Tools/Bulk create users” from the main CKM Toolbar.

2. The admin uploads a CSV File containing the user details as detailed in CSV File Structure above.

3. CKM checks the structure (especially the first row) and displays a table/grid with import data.

   1. If the structure is invalid in any way, an error message is displayed and the user cannot continue.

4. The admin can select the Additional Input Options as detailed above.

5. The admin clicks the Create Users button.

6. CKM creates new users.
   A user is only created if - and only if - the tuple of {first-name, last-name} as well as (the constructed) username, email-address are all unique, i.e. are not currently in use by any existing CKM user account.

   1. CKM creates a new user account for each new user with the provided details including their reviewer and translator availability.

   2. CKM assigns the new users to the selected project (if any) as well as to the specified health domain and profession as applicable.

   3. CKM activates the account immediately or not depending on the respective additional input option.

7. CKM updates existing users in the following way:

   1. If - and only if - user details are provided in the CSV file where the provided triple of {first-name, last-name, email-address} is identical (case-insensitive) to the respective triple of an existing CKM user account [NB: The fixed username of the existing user account is irrelevant here], then the existing user account is updated in the following ways:

      1. Organisation and country are updated for the existing user if provided in the csv file. These details are however not removed if not provided in the CSV file.

      2. The existing user is added to the specified project (if any).

      3. Profession and Health domain are added to the user’s ontology in addition to any existing professions or health domains the user may already have. No documented professions or health domains are removed.

      4. Availability as reviewer, Availability as translator and the current activation status of the existing user account remain unchanged.

      5. The password of the existing user account also remains unchanged.

8. Upon completion of the user creation and update process, CKM displays the following information:

   1. A list of newly created users.

   2. A list of existing users that were updated (matching triple of {first-name, last-name, email-address})

   3. A list of users that could neither be created nor updated for one of the following reasons:

      1. Users where the username (as derived from the first-name and last-name stated in the CSV file) is already taken cannot be created via bulk creation.
         If in addition, the email-address from the CSV file and the email-address of the existing account differ, the user cannot be updated either.
         For example, if a user with username john.smith exists and a user with John as first name and Smith as last name is specified in the CSV file, but with a different email address, the manual process to either create a new user or update the existing user details needs to be followed for this user.

      2. Users from the CSV file where the tuple of {first-name, last-name} is identical (case-insensitive) to the current first-name and last-name of another user cannot be created via bulk creation. If in addition, the email-address from the CSV file and the email-address of the existing account differ, the user cannot be updated either.
         Example: a user named Jane Smith (but with a different username, e.g. jane.maidenname) exists and the CSV File specifies a new Jane Smith with a different email address, the manual process needs to be followed for this user to determine whether this it is the same or a different person.

      3. The provided email address is in use by an existing user - active or not - as primary email address. In this case, the user cannot be created because the primary email address, also used for login purposes, is unique. If then first-name or last-name differ, the user details are not updated either because it cannot safely be assumed that this is intentional and the manual process needs to be followed to create or update the user account.

      4. The provided email address is in use by an existing user - active or not - as its secondary (recovery) email address. In this case, the user is not created. (While this could be done in theory, it is unlikely the desired result, and thus better to avoid this as part of the bulk creation process.) In addition, the existing user account is not updated either because the {first-name, last-name, email-address} triple cannot be identical to the details as provided by the CSV file (due to the fact that primary and secondary email address cannot be identical).

Exceptional Flows

1. *Invalid CSV file:* If the CSV file structure does not provide the necessary columns and details or is not even a CSV file, the import process cannot succeed. The user can try again to upload a different file by returning to Step 2 of the Basic Flow.

2. User classification (profession, health-domain) is specified but does not exist in CKM: It is potentially very inconvenient for the admin if 100’s of users have been created but many or all without being able to specify the user’s profession and/or health domain.
   Therefore, if this is the case for a user, this user is NOT created in the first place. The admin can then correct the CSV file accordingly and restart the Basic Flow. This can be achieved without removing any of the freshly created users from the CSV file because the specified user creation/update process is idempotent (as long as users themselves have not made any changes to their user details in the meantime).

3. Country code is not a valid ISO 3166-1 alpha-2 code: The country code is mandatory for user creation in CKM and thus must be correct. Users without or with an invalid code, thus cannot be created. The admin can then correct the CSV file accordingly and restart the Basic Flow. This can be achieved without removing any of the freshly created users from the CSV file because the specified user creation/update process is idempotent (as long as users themselves have not made any changes to their user details in the meantime).

CKM-1602

Create a new CKM theme based navy blue & teal blue colours

Ocean has developed a new theme for CKM based on navy blue and teal blue colours.

Customers who want to switch to this theme, please contact Ocean.

Note: The background of the top header can be either in navy blue as in the screenshot below or in white.

CKM-1603

Improved User Interface to Update Contributors of archetypes

The user interface to Update Contributors of archetypes has been enhanced in the following ways:

• Use collapsible panels to better show and hide the relevant information

• Information and Warnings are now grouped by Added, Modified, and Unchanged.

• Various colour and layout improvements, including uses of list items and the button to Update Contributors which is in the default blue style.

The Update Contributors functionality is available from the menu of an active branch in the Archetype Revision History

CKM-1615

Improve the layout of the sticky note style (as used on the Dashboard and for editor comments)

Improve the current “sticky note” style used in some places of CKM to look more modern (and less like a sticky-note).

This includes:

• The “Remind to Update Your Profile” as displayed on the top of the Dashboard if applicable.

• The “Would you like to be a reviewer?” widget as displayed on the top of the Dashboard for new users until the have decided to be a reviewer or not.

• The editor comments and special questions in review rounds.

Harmonise with the style used for the right hand side explanation of the “Register as a new user” tab.

CKM-1628

Add caching to the Archetype Data Points report

Currently, compiling the Archetype Data Point report takes a fair while in CKMs with many archetypes. This is not only time-consuming but also quite a drain on the server resources at the time of compiling the report.

To reduce, the (typical) time required to open the most common reports, CKM now caches the latest compiled reports.

This - WHEN cached - reduces the amount of time to close to 0.
The cache needs to be invalidated after changes to archetypes, their status or their projects have been made.

The cache cannot be used when resources in private incubators are possibly selected while the user requesting the report is not a technical admin or Clinical Knowledge Administrator. This is because the report only considers archetypes visible to the user.

CKM-1635

Add support for non top-level pipes in Slot regexes

Slots with regular expression containing non top-level pipes (inside one archetype id expression), e.g.
openEHR-EHR-CLUSTER\.(anatomical_location|anatomical_location_circle)(-a-zA-Z0-9_+)*\.v1|
openEHR-EHR-CLUSTER\.anatomical_location_relative(-a-zA-Z0-9_+)*\.v2

are now considered valid by CKM following the discussion at https://discourse.openehr.org/t/medication-order-medication-details-dosage-ready-for-republication-as-a-new-major-version/2208/10

In addition, the general validity of the regex should be evaluated and reported as a VDFAI error if invalid.

CKM-1642

Update layout/colour of the default button

Update layout/colour of the green default button style. Often, nowadays, a blue colour is used for a default action rather than a green colour and this also harmonises better with the CKM themes.

CKM-1649

"Refresh All" should also check if a resource has been updated and change the revisioning statement if required

The asset version displayed for e.g. an archetype or template will not change with the “Tools/Refresh All” functionality.

This is so because there are various ways in CKM to open a specific version of an archetype or template and then we cannot just update this to the latest. It will however update the Change requests, reviews, etc, revision history i.e. anything that is not directly tied to a particular version of the archetype or template.

What "Refresh All" should do however (and previously didn’t) is to refetch the data used for determining whether the latest revision is displayed or not. If the resource has been updated in between by someone else or e.g. an automated call to CKM’s REST API, the revision at hand is now an older revision and stating that it is the “LATEST REVISION” as in the example tooltip below is now incorrect.

This is now updated on “Refresh All”, and it is thus

• easy to see that this is an outdated asset version and

• easy to select the latest (or latest published) version for example.

CKM-1650

"Refresh All" should also refresh the Manage Users tab and other finetuning

When refreshing the panel via "Tools/Refresh All", an currently open “Manage Users” grid should also be refreshed to take into accounts to updates to the user base.

In addition, the usage information that used to take up an extra line has been transformed into a tooltip next to the heading - only presenting the information when hovered over.

CKM-1661

Project members such as project reviewers should always be able to be invited to review rounds for resources of the project

Project members such as project reviewers should always be able to be invited to review rounds for resources of the project even if the user has otherwise indicated not to be a general reviewer for any resource.

Based on the discussion at https://discourse.openehr.org/t/tooling-support-for-inviting-reviewers/2236/15 it is reasonable to assume that a main task of a project reviewer is to review the resources of the project, even if the user does not want to be involved in reviews for other projects.

This needs to be clear in the selection process of whether a user would like to serve as a reviewer: The user can either select to be available as a reviewer for any applicable not blocked resource or only for any resources of the user's project.

Note: Independent of this, a user can of course decline or ignore any invitation.

This also includes the following work:

• Wordsmithing for reviewer availability messages, separated to avoid repetition in localisation files.

• Add tooltip to Available as reviewer to explain this behaviour.

• Add explanation to the Invite More Users Tab when creating a review round.

While this is intuitive, the exact rules of when a user can be invited are complex and have been summarised in the following table:

CKM-1670

The "Active Review Round already exists" warning should be based on whether the previous review round has been closed or not, not based on the completion deadline

Previously, when setting up a new review round for a resource while a review round of the same type exists already for the resource to be reviewed, the initiator was warned that an active review round exists IF the completion deadline hasn't yet passed.

However, it seems more reasonable to trigger the warning if the review round is still active (i.e. not closed by the editors). This is because the completion deadline may have passed a long time ago in some cases, but the review round may still be accepting reviews.

Therefore: The "Active Review Round already exists warning" should be triggered based on whether any previous review round of the same type has been closed or not (and not whether the completion deadline has passed or not).

Also, this warning was previously not currently triggered at all for template reviews. It should be triggered in the same way as for all other review types.

CKM-1676

Upgrade the internal terminology client to LOINC 2.72

Upgrade the internal terminology client to LOINC 2.72 (February 2022) release.

Note: Customer-specific configured terminology servers with a FHIR API can always be upgraded to use various terminologies at any time and take precedence. However, this is used to ensure all CKM instances can display LOINC term bindings etc.

CKM-1678

Update template status via the REST API

In CKM instances with for example well over 20,000 templates - based on an advanced tool-chain - it is important to automate the template publishing process.

This issues introduces an extension to CKM’s REST API to be able to update the resource status of a template.

PUT /templates/{cid-template}/status

with parameter status=PUBLISHED (or any other state that is possible as a next step, depending on the templates current state).

The proceed-if-active-branches parameter can be used to control whether to proceed with the status update even if the resource has active branches. This is because before some status changes, it may be desirable to clean up (commit, resolve, reject or delete) any active branches. Defaults to false.

CKM-1680

Separate the service to "delete an user and delete all users" into two separate services with different privileges

Previously, there was one service that enabled the deletion (“unregistering”) of a particular user. This service featured an additional option to delete all normal users. This exists for administrative purposes only and would only ever be used by Ocean admins.

To clarify this and also to improve the rights management for this service, this service has now been split into two separate services. Only the first service is available to non Ocean admins in the first place.

CKM-1683

Extend CKM's resilient mail sending service to also try to recover if the email was initially advised to be sent asynchronously

CKM's resilient mail sending service always tries to recover and resend emails if the configured mail server is temporarily down and did not accept any mail for whatever reasons.

However, previously this only applied to emails that were advised to be sent synchronously initially. While this is the default for nearly all cases (because it enables feedback to the users if emails could not be sent), sending emails synchronously is fairly slow.

For bulk operations requiring the sending of mail (like CKM-1554: Bulk Create/Update Users via CSV File Upload) synchronously sending the emails is far too slow.

Therefore the same functionality to recover from a mail server that is temporarily down has been extended to emails that were initially advised to be sent asynchronously.

CKM-1685

Improve Welcome Emails

Improve the Welcome emails to new users - with or without activation required.

• Add CKM instance abbreviation

• Clean up and finetune the English welcome and activation text

CKM-1686

Clean up the Template Resource Centre sections

Clean up some aspects of the resource centre for templates, such as the CDA related group of sections which is likely unnecessary nowadays.

In detail:

1. Remove the CDA Implementation Guide section.

2. Remove the CDA Schematron section.

3. Integrate the “Specification” section with the Reference and Design Documentation. (Remove the Specification section: the single use of this section was for a rejected example template which has been removed completely as advised by the customer rather than carried over to Reference and Design Documentation.)

CKM-1691

Improve the visual distinction of secondary headers in CKM tabs and harmonise the first header underneath the resource icon toolbar

Secondary headers are used in some places for further grouping in addition to the first heading, for example in the Resource Centre for Archetypes and Templates.

However, the visual distinction between the first header underneath the icon toolbar (for a resource) and the secondary header was not always clear. This has been improved and harmonised.

This applies to the headers underneath the icon toolbar for an archetype, template, termset or release set, as well as projects and subdomains.

CKM-1692

Support JSON as Sample Data in the Resource Centres

Sample data related to archetypes or templates can be uploaded to the Resource Centre of the resource.

However, this is currently limited to XML whereas sample data nowadays often will be in json format.

The advantage of using XML is that XML Sample Data can be displayed using various (configurable) transforms. This is potentially quite powerful, but has rarely been used (see e.g. https://ckm.openehr.org/ckm/archetypes/1013.1.130/resourcecentre and click on Render Sample Data for the uploaded Sample Data for an example.)

Therefore:

1. The Sample Data Section of the Resource Centres for archetypes and templates has been extended to support the upload of JSON sample data as well.

2. For JSON sample data, a generic display of the sample data in JSON has been added. For XML sample data, the status quo is preserved.

CKM-1694

Archetypes: Fold the ADL and XML tabs into one and finetune icons and texts

For archetypes, fold the ADL and XML tabs into one tab with archetype serialisations (ADL 1.4, ADL2, XML for now).

The XML (1.4) tab is rarely used and combining the serialisation formats under one tab reduces the number of icons in the archetype toolbar. It also enables easier integration of more serialisations in the future.

The ADL tab already has two options 1.4 and 2.0 and having the xml serialisation as part of the same tab seems reasonable and helpful.

Also: Improve the ADL icon (but keep as ADL as the most common serialisation format) and the (related) ADL Validation Report icon) and update the tooltip and context menu text to “ADL / XML”.

CKM-1699

Various finetuning of subdomain related displays

• Project and Subdomain Members tab: Harmonise the width of the initial instruction, which should not stretch across the complete page.

• Finetune the display of the Projects per subdomain panel (available for each Subdomain): layout, css, icon, button panel, margins/paddings.

• Finetune the display of the “created by” notice on the subdomain panel (for admins) and harmonise with the display of the similar message on the project dashboard, add tooltip and link to user, use the user’s current full name for display purposes instead of the login-name.

CKM-1711

Minor finetuning of Project and Subdomain Export panels

Minor finetuning of Project and Subdomain Export panels to ensure that after clicking on the Export button,

• the Export button is temporarily disabled,

• the cursor is temporarily changed to a wait cursor

• A message is temporarily displayed to indicate that the zip file is being generated and retrieved.

This is so that a user get better feedback that the button has been clicked. Also, add an icon to the export buttons.

(Note however: It is technically not possible to determine the exact point in time when the zip file is eventually presented to the user for storing/opening, therefore the time to end these temporary actions can just be estimated.)

CKM-1712

Harmonise heights in Review Round creation panel in all languages

Depending on the language, a few of the various panels that are displayed during the steps of setting up a new review round or modifying an existing Review Round had different heights. The heights have now been harmonised.

CKM-1715

Increase the height for the "Available as reviewer" and "Available as translator" and the "User roles" tab within the User Profile Panel & other minor finetuning

Up to now, all tabs of the User Profile had the same height.

However, the 2nd, 3rd and 4th tab (reviewer availability, translator availability and the user roles tab) would benefit from making better use of the available height, especially when selecting health domain, professions or languages.

Therefore, their height is now change accordingly on tab change.

Also: Finetune wording and tab paddings.

CKM-1717

Improve usage of separators in the main menu for the various roles

Improve usage of separators in the main menu for the various roles by adding functionality to hide non-relevant the separators for non-editors. This way we can add some additional dotted separators but hide them for users where they are not necessary.

CKM-1748

Add the semantic version numbers of the latest and the latest published version to the REST API Archetypes listing

CKM’s REST API can list archetypes as well as retrieve the main data for one archetype in a particular version.

Currently a part of the retrieved meta data, the asset version (e.g. 9) as well as the semantic revision number (e.g. 1.0.1-alpha) of the retrieved archetype is included (if it exists), see line 14.

Also, the asset versions of the LATEST and LATEST PUBLISHED archetype asset version are retrieved, see line 10 and line 11.

In addition, we should also retrieve the semantic revision numbers corresponding to the LATEST and LATEST PUBLISHED archetype asset versions: see line 15 +16 in the json below.

CKM-1751

Update openEHR logos

Update the openEHR logos where used in CKM to the latest version of the logo / favicon; see

This affects the AboutPanel in CKM in the Powered by Section in all CKM instances.

In addition, there are three places where the logo may be used by various CKM instances:

• In the header, top left as the instance logo

• In the About Panel - as the big instance logo

• As the favicon

CKM-1752

Add CKM logo to About Panel, About Menu Item and (randomly) Dashboard

Add the new CKM logo to the About Panel in CKM and the corresponding menu item. Restructure Layout of the About Panel to accommodate the logo and make better use of the available space.

Also add the logo to the Dashboard as one of the randomly chosen community images. For a dark theme like the ocean blue, the dark version of the logo is used, for the lighter themes (blue, gray) the light version is used.

CKM-1581

Uploading an archetype must only be allowed if all relevant parts are available for all languages; an appropriate validation warning should be displayed & the archetype's Status Report must be displayed correctly for such archetypes

Uploading an archetype should only be allowed if for all translations present in the archetype, all relevant parts are available.

That means, the archetype must have

• language/translations

• description/details

• ontology/term_definitions

for each translation language included.

If for example an archetype is uploaded to CKM that has the following characteristic/problem:

• The archetype contains an ontology/term_definitions section for a translation language A

but:

• The archetype does not contain a corresponding language/translations section for translation language A, or

• The archetype does not contain a corresponding description/details section for translation language A

then this archetype should not be imported/updated.

For existing archetypes, there should be a validation warning.

One example where this problem has occurred in production is the diagnosis archetype in Rev 36 . Chinese (zn) has been added as language, however the description/details have been omitted.

For existing archetypes with this problem, this also prevented the comprehensive status report to be displayed for this archetype. This has been fixed as part of this issues as well.

CKM-1611

The comprehensive status report displayed on the resource status page for an archetype or template may have (theoretically) presented wrong results if the underlying service is called with different parameters at the exact same time

Theoretically, the comprehensive status report displayed on the resource status page for an archetype or template may have presented wrong results if a huge number of different requests of this type are made to the server simultaneously. Under very heavy load if the service to compile the comprehensive status report is called multiple times at nearly the same millisecond with different parameters, runtime conditions may have caused three variables to be updated with new values before they have been used for the previous call.

Note: This has - to the best of Ocean’s knowledge - never been experienced in a production system, but has been fixed nonetheless.

CKM-1614

Data Points Report: When selecting a subdomain AND a project type, the project type is not honoured in selecting the archetypes to be assessed for data points

In the Data Points Report available from the Reports Menu:
When a subdomain AND a project type are selected, the project type was not honoured in selecting the archetypes to be assessed for data points.

For example:

1. Open the Data Points Report from the Reports Menu.

2. From the Options: Select a Subdomain (assuming the CKM uses more than one subdomain, otherwise this issue cannot be reproduced)

3. From the Options: Select "All Projects" or "All Incubators"

4. Click the View Button

Whether "All projects" or "All incubators" is selected doesn't make a difference to the results in CKM 1.17.0. This has been fixed.

NB: Selecting the appropriate resources states (e.g. INITIAL for incubators), works correctly and can be used as an alternative.

CKM-1618

The default form view for a template does not display date/time widgets for an interval of date time

In a template that uses a DV_INTERVAL<DV_DATE_TIME> datatype or any of the other Date intervals, the template’s form-based view does not show widgets to select the lower and upper date time.

Instead only "lower:" and "Upper:" are shown:

See the example of “Gültigkeitszeitraum” at:

CKM-1626

Improve the error message on uploading an archetype as part of creating or updating an archetype proposal if the archetype is not parsable

When uploading an archetype as part of creating or updating an archetype proposal and the archetype is not parsable, CKM presents the user with an error message that the archetype proposal cannot be created and shows the verbatim parsing error that was encountered.

While it is desired behaviour that an unparsable archetype is not accepted, the error message should clearly state that this is a Parsing Error.
Without this, error messages like

Encountered " "sd "" at line 37, column 61. Was expecting: "}" ...

sound very cryptic.

Therefore, extend the error message as follows:

The archetype cannot be processed because it has parsing errors.
Archetype Parse Error: Encountered " "sd "" at line 37, column 61. Was expecting: "}" ...

CKM-1630

Archetype Statistics Report: Local vs Remote archetype chart is not displayed for projects, but heading and legend

1. Open Archetype Statistics Report from the Reports menu

2. Select a project without selecting a subdomain.

→ No local vs remote archetype chart is displayed. However, the related heading and chart legend are displayed.

This needs to be removed completely (including heading and legend) if a project is selected.

CKM-1631

Not all archetypes may be returned for consumption - e.g. for the Archetype Data Points Report

NB: This issue has never occurred in a CKM production system.

Only a subset of applicable archetypes may be returned using a generic query to retrieve archetypes. This can manifest itself for example in the Archetype Data Points Report where not all archetypes may have been considered.

This problem has been traced to a complex indexing problem. While recreating the indices did not have an effect, manually destroying and then creating the indices again was a possible workaround. In addition, the performance of these queries with indices enabled was poor.

Ocean has worked with the vendor of the database system to rectify this issue. At the same time the performance of these queries has been significantly improved:

• Fixed a problem where the evaluation of "A and NOT(B)" where B was an expressinon of the form "xpath hasno value" might return different numbers or results when indexing was enabled or not.

• XODB: Significantly improved the performance of evaluation of some queries which referred to multiple keys (a range of keys) in a range query.

Note, that this issue was introduced in a version of that system that was never used by a CKM production system and has been fixed for the target release of the database system for CKM 1.18.0, so that this was never a problem in any CKM production system.

CKM-1632

Improve the "No resources found" message when certain states are selected in the Archetype Data Points panel

The "No resources found" message has been improved in the Archetype Data Points panel.

Previously, when certain lifecycle states were selected in the Archetype Data Points panel and the selection yielded no result, the error message just stated that the project (or subdomain) does not contain any archetypes, but without any reference to the states under consideration. This was misleading and the “No resources found” message now includes the selected states where necessary.

CKM-1634

Checked out resource icon is missing in the tab headers

1. Log in as editor or admin

2. Select Archetypes/Revise Archetype/Check-out archetype for modification

→ The tab header of the newly opened CKM tab should show the Checkout icon, but was not.

Likewise for Templates and Termsets.

CKM-1643

Creating a new review round when the respective part of the resource is marked as published (/completed) fails only at the very end

Creating a new review round when the respective part of the resource is marked as published (/completed for translations) fails only at the very end.

To reproduce, create a new content review round for a currently PUBLISHED archetype.

This will fail at the very end when creating the review round because the archetype is currently not under reassessment. Similar for translation reviews etc.

While it is correct that the resource cannot be reviewed in this case, the warning should occur at the beginning when creating the review round and selecting the archetype to avoid (potentially significant) unnecessary effort.

CKM-1645

If an editor submits a translation review, the Dashboard is not updated with the new review count on the Editor’s review round Dashboard (one more completed)

Previously, if an editor submitted a translation review, the dashboard was not updated with the new review count on the Editor’s review round Dashboard (one more completed). Now, the dashboard is updated immediately to avoid possible confusion whether the review was completed successfully or not.

CKM-1651

The drag and drop functionality to upload an archetype may not always work correctly at all places in CKM in all browsers

When creating or updating an archetype proposal or a change request, the drag and drop functionality to upload an archetype file, previously did not always work correctly in all browsers:

When an archetype file was dropped onto the drop area of an Archetype Proposal, CKM may incorrectly have claimed that no (mandatory) file name had been provided yet and thus refused to create or update the Archetype Proposal.

Likewise, when a file was dropped onto the drop area of a Change Request, CKM may have incorrectly ignored the (optional) file and then created the Change Request without the attached file.

The same problem occurred when dragging and dropping a document review file during the setup process of an order template review round.

As a workaround, it has always been possible to use the File Dialogue to upload the file without problems (instead of using the drag and drop functionality).

CKM-1652

SVG images should be supported as the default user image and an SVG image used that works across themes

The user default image that is displayed for a user when a user has not uploaded a personal image, should support the usage of svg images. This is relevant for the user image displayed in the logged-in user’s Dashboard, the login toolbar, the user profile and the discussions.

This is required to ensure that the image works well across different CKM themes. The default user image has been recreated to be an SVG file.

CKM-1657

Reverting a remote archetype to a previous version may cause a configured git repository to list this file as part of the local archetypes until next full export

Previously, reverting a remote archetype to a previous version may have caused a configured git repository to list this file as part of the local archetypes until next full export.

It must however be exported to a directory remote/<remote-subdomain-namespace>/archetypes

Steps to reproduce:

1. Import an archetype form remote, selecting asset version 1.
   This must be an archetype that has more than one asset version already.

   • → Git export correctly exports the archetype to the remote dir as a new file.

2. Update this archetype from remote, selecting asset version 2 this time.

   • → Git export correctly exports the archetype to the remote directory as an updated file.

3. In the archetype’s revision history: revert the archetype to revision 1

   • → Git export incorrectly exports the archetype to the local directory as a new file (e.g. local/archetypes/cluster/<archetype-id>

   • → Git export leaves the archetype file in the remote directory untouched. (This should be updated instead).

CKM-1659

Improve error message on uploading an archetype as part of creating or updating a Change Request with an archetype attached if the archetype is not parsable

When (optionally) uploading an archetype as part of creating or updating a Change Request and the archetype is not parsable, CKM presents the user with an error message that the change request cannot be created and shows the verbatim parsing error that was encountered.

While it is desired behaviour that an unparsable archetype is not accepted, the error message should clearly state that this is a Parsing Error.
Without this, error messages like for example

Encountered " "sd "" at line 37, column 61. Was expecting: "}" ...

sound very cryptic.

The archetype cannot be processed because it has parsing errors.
Archetype Parse Error: Encountered " "sd "" at line 37, column 61. Was expecting: "}" ...

CKM-1660

Order template reviews: Increase resilience of the document review json file upload

For order template reviews (available for order templates only), the document review json file upload should provide proper error messages

• if the file is not a json file at all, or

• if the file is correct json but does not contain any document review sections at all.

CKM-1664

"View uploaded branch" option does not open the branch under some circumstances (e.g. after submitting a translation from one branch to another)

The "View uploaded branch" option does not open the updated branch for example after submitting a translation from one branch to another.

This problem occurs for archetypes that have been previously published.

Steps:

1. Find an archetype that is currently published or has been (i.e. is being reassessed at the moment)

2. Upload a changed archetype to a branch of that archetype or submit a translation from one branch to another

3. Select the “View uploaded branch” option

4. Click the button to continue

→ The submission succeeds but before this fix, the archetype branch did not open in a new CKM tab.

CKM-1665

Email reviewer window: Buttons may be not fully visible and there should be a sufficient minimum height for the text area even if the user is using a very low height for the browser window

In the email reviewer window, the buttons may be not fully visible, depending on language (here Norwegian).

Also, there should be a sufficient minimum height for the rich text area even if the user is using a very low height for the browser window.

CKM-1666

Initial (setup) Git export may fail

The initial (setup) complete git export required on setup may fail with an error message like the following:

Git export failed: Invalid path: //mote/no.nasjonalikt/archetypes/cluster/openEHR-EHR-CLUSTER.microscopy_renal_biopsy_non_neoplastic.v0.adl
org.eclipse.jgit.dircache.InvalidPathException: Invalid path: //mote/no.nasjonalikt/archetypes/cluster/openEHR-EHR-CLUSTER.microscopy_renal_biopsy_non_neoplastic.v0.adl

Note that the path is specified to be "remote/..." not "mote/..." and CKM here just adds all files using a "." filepattern.

After some investigation, it appears that this is a problem in the jgit library used in CKM at present in this particular case.

Issue resolved by upgrading to the latest library: CKM-1667: Upgrade JGit to latest (6.0) version

Note: This problem only occurs on setup of a new git repository and only after some rework of related functionality in CKM and has never occurred in any CKM production system.

CKM-1671

When reverting a REMOTE archetype to a previous revision, must exactly match the remote resource and its remote resource status at that revision

When reverting a REMOTE archetype to a previous revisions, the reversed archetype must exactly match the remote resource and its remote resource status at that revision.

In particular, the following scenarios need to be handled in the following ways to ensure this is the case:

1. If the remote archetype is reverted to a very old archetype that uses a .v1 extension despite being a DRAFT, this needs to be maintained - CKM is simply mirroring the remote repository here.

2. The Resource Status of the remote archetype at the version reverted to must be applied again. (This is different to the handling of local archetypes where we either keep the current resource status or transition it following its natural lifecycle - e.g. from PUBLISHED to REASSESS_DRAFT if necessary). LOCAL archetypes that are being reverted must - in addition to appropriately following the publication lifecycle - also receive a new build id, adapt a legacy v1 draft archetype to v0, etc., whereas a remote archetype must use whatever the remote archetype provided. Again CKM is just mirroring the remote repository.

3. It cannot be allowed to revert the archetype to a revision that changes the major version of this archetype.

CKM-1672

Using a ", ', <, or | in the name of a class in the flexible ontology can lead to some downstream problems when managing or searching the ontology

Using a ", ', < or | in the name of a class in the flexible ontology can lead to some downstream problems when managing the ontology, searching it or classifying users or resources:

1. Cannot delete the class again via the UI (error message for ' and no message at all for " because of next point)

2. For " the right click does not work when managing the ontology and therefore no action is possible.

3. At least for ", it is not possible to assign a resource to a class that contains this char.

4. The pipe symbol | is used as replacement character for the forward slash / (which in turn is used for class hierarchies) and thus cannot exist independently.

Therefore:

• Prevent the creation of classes in the ontology with any of the following characters: " < |

• Ensure that ' is properly supported (create, update, move, delete) as well as for classifying users and resources.

CKM-1675

Archetype Validation Report: The List Box to filter by validation error type doesn't trigger on change

Archetype Validation Report: The List Box to filter by validation error type (that is shown after all validation errors have been displayed) doesn't trigger on change any more. Any validation error can be selected, but the displayed validation errors are not filtered accordingly.

CKM-1679

First login after project-based user registration: If user opts to set the Preferred View to the project using the popup this fails

On the First login after Project-based user registration: If the user opts to set the Preferred View (in the left hand accordion panel) to the project using the pop-up, this previously failed with an error message that the user configuration could not be stored.

Steps to Reproduce:

• Use the project-based registration link available as editor from the Mange Project tab of the project to create a new user.

• Follow the process to activate account and login as the new user.

• The user has already been added to the project and is now asked if the user wants to set the Preferred View to the project.

• If the user chooses Yes, an error message popped up explaining that the user configuration could not be stored. Instead, the Preferred View should be set to the project.

CKM-1682

CKM is reloading itself on user login under specific conditions

Under certain conditions, CKM previously reloaded itself upon user login and was then calling (restarting) the module again.

While this did not have a direct impact on the display, it should obviously avoided for performance reasons as well as any potential side effects.

CKM-1688

The "All Resources" left hand accordion panel may show wrong results if used in a certain way in CKMs with multiple subdomains

Steps to reproduce

1. Select a Project

2. Select a Subdomain

Upon selection of the subdomain in Step 2, the selected project from Step 1 is cleared from the Project Select Box.
This makes sense because the project may not be part of the subdomain. However, it does not seem to be cleared when retrieving the archetypes and templates of the newly selected subdomain. This previously resulted in the more specific project selection taking precedence and thus project resources were displayed instead of the subdomain resources.

CKM-1689

Cannot open a subdomain as not logged in user

A user that is not logged in cannot open a subdomain, e.g. from the Projects/Open Subdomain menu or via a link from a project to that subdomain.

From a user’s perspective, the subdomain is simply not opened, i.e. no new CKM tab is opened for the subdomain.

However, the subdomain should open (for display purposes) even if the user is not logged in.

CKM-1695

A more specialised translation language may erroneously be used instead of the more general requested language for display purposes under certain circumstances

Under certain circumstances, a more specialised translation language may erroneously be used instead of the more general language requested for display purposes (archetype html, archetype printable html, archetype mind map).

This can happen if all of the following conditions are met:

1. A language is requested without a region: e.g. “es”, AND

2. The archetype contains a translation “es” and also a more specialised (“regional”) translation such as “es-ar”, AND

3. The “regional” translation (es-ar) is listed before the general translation (es) in the archetype ADL.

For an example, see the Problem/Diagnosis archetype in the international CKM instance: and select Spanish (Spain) as language - this erroneously picks up Spanish (Argentina) due to the described issue.

CKM-1698

Reject import of archetypes, templates, termsets with extremely long export filenames

File names used are typically based on the archetype id, the template name, and the termset query short concept name.

Unix only supports file names up to 255 characters (bytes). File names > 255 characters can therefore not be stored on the local file system e.g. for git export.

Therefore, restrict archetype, template and termset primary names and thus file names to a maximum of 255 chars.

Note: For downloading from CKM, the maximum file name length is (and has always been) considerably less already. This is considering that a normal Windows system - without long paths explicitly enabled - only supports 260 chars for the whole path. However, truncating the file name on download is less problematic than for a git export - because this may potentially lead to duplicate file names.

Therefore:

• Reject the import of any archetypes with an archetype id longer than 255 chars.

• Reject the import of any template with template name longer than 255 chars.

• Reject the import of any termset with a termset short concept name longer than 255 chars.

CKM-1701

Implicit value sets (from CodeSystems) are not updated if cached

An implicit value set (defined as part of the CodeSystem) will be cached in CKM.

However, in contrast to explicit value sets, the <meta> element with versionId and lastUpdated time may not be populated when retrieved as (expanded) value set.

<meta> <versionId value="1"/> <lastUpdated value="2021-10-11T13:15:57.424+00:00"/> <profile value=http://hl7.org/fhir/StructureDefinition/shareablevalueset/></meta>

 The „lastUpdated" field is used by CKM to determine if the cache is still current.

If the field is not populated, CKM currently assumes - in this case too optimistically - that the cached value set is still up-to-date.

This may obviously be a problem if the underlying CodeSystem changes. More importantly maybe, it is also a problem if the CodeSystem and its implicit value set have not been uploaded to the Terminology Server (and fully initialised for large code systems) BEFORE they are requested the first time by CKM via an archetype or template.

The workaround is to manually delete CKM's Value Set Cache.

Without the lastUpdated field, CKM cannot reliably determine whether the Value Set has changed in some way. Therefore, it is now heuristically determined, whether the cached value set should be considered as outdated in this special case.

CKM-1704

Editor should ONLY be notified that it is not necessary to manually set the status to team review if the current status is a state without an active review round

On manually changing the resource status to TEAM REVIEW, an editor should ONLY be notified that it is not necessary to manually set the status to team review if the current status is a state without an active review round (draft, reassess draft, published), but not if the current state is a review suspended state. In that case, the manual change is required and thus the warning would be misleading.

CKM-1706

Release Set Editors should be able to update the status of a release set from the status page of a release set

Release Set Editor should be able to update the status of a release set from the status page of a release set.

Currently, this only displays the current status, but a Release Set Editor - who doesn’t happen to be CKA or admin at the same time - cannot update the status from that page.

(As a workaround, a Release Set Editor can still update the status from the context menu of the resource).

CKM-1710

When setting up a new review round, the height available for the Review Round Selection panel may be insufficient, especially if no translation or terminology exists

When setting up a new review round, there is insufficient height for the Review Round Selection panel, especially if no translation or terminology exists. This is caused by the improved display of the error panel as well as the improved font for the display name of the archetype (or other resource).

CKM-1713

An initial revision of an archetype that was referenced from remote and later internalised cannot be opened via its direct link

An initial revision of an archetype that was referenced from remote and later internalised cannot be opened via its direct link.

For example: was originally referenced from the remote Nehta CKM instance, and was later ‘internalised’ as a fork of that archetype.

Using the direct link to the first revision (-> where it was still referenced from remote), leads to a blank CKM tab in CKM.
Opening it in another way works fine.

Step to reproduce:

1. Import an archetype from a remote subdomain ( → Asset revision 1)

2. Internalise this archetype ( Go to revision history, click on the Details button, select Internalise Archetype) ( → Asset revision 2)

3. Go to the first revision of the archetype (this works fine)

4. Copy the direct link from the browser bar

5. Use the copied direct link to open CKM again.

→ Previously this produced a blank CKM tab. Now the archetype is displayed in the tab as expected.

CKM-1716

The panel to add additional reviewers to existing review rounds should be a little smaller so that it fits on the screen without scrollbars

The panel to add additional reviewers to existing review rounds (Reviews/Invite additional reviewers) should be a little smaller so that it fits on the screen without scrollbars, assuming a reasonable height available. Also: finetune some other widths in the panel.

With appropriate rights, this panel is available via Reviews/Invite Additional Reviewers.

CKM-1719

Minor CSS Style sheet / themes problems

Fix some minor problems in various CKM style sheets, namely the main CKM style sheet and the various theme related style sheets.

CKM-1412

[Internal] Streamline the review invitation process

Due to various additions and changes over the years, the internal process to invite reviewers to a review round was quite complex and convoluted. For example, originally, review invitations used to work without an explicit review round.

The review invitation services on the backend have been reworked and streamlined in the following ways:

• A complete list (set) of users to be invited is compiled first. This includes users to be invited via name and users invited via their role within a CKM project, etc. Only then the review invitations are sent out, rather than for each project.

• The backend service to create review invitations for a project (oi.review.invitations.create.for.project) can then be removed - it currently creates the invitations but does not attach them to a review round, and for this reason should never be called directly anyway.

• The service to create one review invitation oi.review.invitation.create can then also be removed and made available from within the PReviewInvitation class, so that it can be called when creating or updating a review round - which nowadays includes the review invitations.

CKM-1575

[Internal] Remove the very experimental ADL 1.4 to ADL 2 converter

Remove the very experimental ADL 1.4 to ADL 2 converter which is no longer needed to be developed any further because this functionality is now covered via the fabulous Archie library.

CKM-1584

[Internal] Rework XSLTTransformer, AssetUtils, StringUtilsCore, FileUtils, GitConnector, PictureServer, FileReceiver, etc. to harmonise InputStream consumption, File handling, zip file handling, TimerTasks etc. taking advantage of newer Java syntax

Rework many files and services such as XSLTTransformer, AssetUtils, StringUtilsCore, FileUtils, GitConnector, PictureServer, FileReceiver, etc. to harmonise InputStream consumption, File handling etc. taking advantage of newer Java syntax.

Currently the XSLTTransformer uses its own InputStream handling to extract the XSLT as a String.
AssetUtils does similar things, using Apache's IOUtils. Harmonised, removing the need for IOUtils.

Update Input Stream handling to modern syntax with try-with-resources and Java 9, for example:
String contents = new String(stream.readAllBytes(), StandardCharsets.UTF_8);

This has now be moved to the shared StringUtilsCore as we have now left the old integrated Jetty behind, preventing this syntax in the middle tier.

Also in the XSLTTransformer, there is a getPrettyPrintedXML which uses getContentForCid and convertStreamToString.

Use java.nio’s StandardCharset constants for UTF-8 etc.

Use java.nio.Path etc. as a replacement for Files where applicable and doable, which provides a more robust API.

Use java.nio.Files Utilities etc. to streamline InputStream, OutputStream, File reading and writing activitities.

Use a TimerTask for scheduling file deletions after upload where required.

Upgrade ZIP utilities to make use of Path and Files nio - e.g. Files.walkFileTree, Files.copy, Path, FileVisitResult, etc.

CKM-1587

[Internal] Replace use of deprecated Apache's CharEncoding with native Java nio charsets

Java 7 introduced StandardCharsets (especially for UTF-8), which defines these constants as Charset objects. Consequently, Apache's CharEncoding has been deprecated. Where still used in CKM, this is being replaced with the native Java nio.charset.StandardCharset.

CKM-1589

[Internal] Upgrade to target Mediaflux version for CKM 1.18.0

Upgrade to the target Mediaflux version for CKM 1.18.0, namely version 4.13.032.

CKM-1594

Finetune size and insertion condition of middle-tier archetype-object cache and rework its implementation so that the LRU Cache is easier reusable for other object types such as templates

Finetune size and insertion condition of middle-tier archetype-object cache and rework its implementation so that the LRU Cache is easier reusable for other cache types

1. Increase number of cached elements to 50

2. Decrease minimum creation time required to be added to the cache. (The overarching idea is that the archetypes that are more time-consuming to construct end up in the cache)

3. Rework using a Cachable interface and a separate LRUCache that is used by the ArchetypeObjectCache to be able to reuse the LRUCache for other objects such a templates in the future.

CKM-1610

[Internal] A wide variety of syntax updates & improvements, package restructuring & comment improvements

This issue contains a wide variery of syntax updates and improvements, many source code comment improvements, some package restructuring.

Some of the syntax updates have simply not been possible at the time of originally writing. Some have only been made possible recently for example upon removing the dependency on the outdated inbuilt Jetty, and upgrading to GWT 2.9 with enhanced Java syntax support.

Changes includes for example:

• Consistent use of interfaces, esp. for List and Map. (This wasn’t possible / didn’t properly work in an early GWT version and should be used consistently now (also in the backend).

• Avoid repetition of types when e.g. constructing HashMaps, ArrayLists etc → Diamond syntax. Note however, that GWT, despite generally supporting Java 9 syntax now, does not support this for anonymous classes like used for the AsyncCallbacks.

• Reformat source code and check for problems (linebreaks of comments, method signatures etc.)

• A multitude of comment and code formatting improvements

• Create and use getter for the executor provided by the GeneralMediafluxObject and make it private

• Various restructuring of packages in the backend

• Many changes improving syntax, comments, formatting, harmonising of parameter order and naming, with a focus on the AssetUtils class.

• Extract an separate LockKeyProvider from the AssetUtils.

• Extract a separate SchemeUtils class from AssetUtils.

• Cleanup and streamline various panels like the EmailSendPanel, ProjectMultipleSelectAndRolesSelectPanel, ReviewRoundCreateOrModifiyPanel.

• Give more control to the (renamed) ProjectAndRolesMultipleSelectPanel to return its selections directly rather than exposing all checkboxes etc.

• Rewrite/cleanup CKM’s DocumentationServer to make use of more modern Java syntax for output stream etc.

• Restructure, cleanup and harmonise file serving and transforms, incl. for zip files for all resources and transforms.

• Clean up of Resource Main Panels and Share with Colleague panels in preparation for CKM-1694 Ready For Test - Archetypes: Fold the ADL and XML tabs into one

• Remove hopefully completely obsolete Firefox on Vista warning for downloads

• Harmonise Resource Status enums

• Harmonise the main toolbar code

CKM-1617

[Internal] Improve CKM Connection handling and pooling

Improve Connection handling and pooling by

• replacing a legacy HashTable pool in MFConnector can be replaced with a ConcurrentHashMap

• removing the need for a separate map with user logins mapped to sessions

• Returning a dedicated MFSession object rather than a list of strings with username, sessionId, and optional permSessionID or at least a Triple.

• Using a composition approach wrapping the mediaflux connection and forwarding required method. One reason for this is to be able to be able to implement the AutoCloseable interface and use try-with-resources instead of a finalize block where required. (Major rework here)

• Rework the session and connection synchronization and creation (Streamline, avoid unnecessary synchronization to improve connection speed)

• Increase the number of not logged in user sessions to increase speed, especially when multiple users are loading ckm simultaneously.

• Rework the MFConnector as Singleton to avoid frequent and rather unnecessary instantiation each time.

• Fix a couple of - minor - connection closing issues and missing session ids provided for calls.

• Improve connection clean up on expiry of user sessions where the user just navigates away from cKM or closes the browser completely etc., rather than logging out.

• Remove the need for a separate server call to test if connections (licences) are available by merging this with the call to get the freshly logged in user details

CKM-1627

[Internal] Make mediaflux package build script independent of concrete javac file path

Current mediaflux plugin package build script uses an (optional, but currently activated) dedicated path for compilation purposes. This should at least by default just use the default - which needs to be set as part of the PATH variable anyway.

CKM-1636

[Internal] Introduce CSS variables to the gray theme style sheet

Introduce and use CSS variables in the gray CKM theme. This is to make future changes easier and provides a solid basis for creating a new theme (see CKM-1602: Create a new CKM theme based navy blue & teal blue colours)

Note: this is possible because all relevant browsers now support CSS variables.

CKM-1637

[Internal] Rework and extend theme handling

Rework/Reorganise the theme configuration and theme selection.

CKM-1638

[Internal] Remove the CKM internal Base64Coder and ObjectSerialiser

The Base64Coder and ObjectSerialiser are no longer in active use and can be removed.

If anything like this is needed in the future, the corresponding native Base64 encoder and decoder could simply be used now.

CKM-1641

[Internal] Harmonise classpath files, cleanup .gitignore and the build script

Harmonise classpath files to ensure usage across system during development/compilation.

Also clean up some obsolete file paths in .gitignore

CKM-1646

Upgrade to latest stable target Archie version for CKM 1.18.0

Upgrade the Archie version used by CKM to the latest stable Archie version for CKM 1.18.0 → 2.0.1.

CKM-1655

Rework the Subdomain Export Service to streamline (and thus significantly improve the speed of) the export

The Subdomain Export Service has been streamlined.

Previously, resources were retrieved and exported more than once if they were referenced by several projects of the subdomain.
While this produces correct results, in larger subdomains with many projects often referencing the same resources this considerably slows down the creation of the subdomain export zip file.

Also, some internal query optimisation (splitting the relevant query into two parts that can make better use of indices) has very significantly increased the performance of this service.

As a result - for a typical large subdomain with many projects - the zip file can be presented to the user about 20x faster than before.

CKM-1662

[Internal] Create reusable common Disclosure Panel

With increased use of Disclosure Panels in CKM, the Disclosure Panels should be based on a common reusable and consistently styled CKM widget.

CKM-1663

Upgrade to latest Java Reference Implementation (Parser, Validator, Serialisers)

Upgrade to latest Java Reference Implementation libraries (Parser, Validator, Serialisers).

CKM-1667

Upgrade JGit to latest (6.0) version

Upgrade JGit to latest (6.0) version.

Refer to CKM-1666: Initial (setup) Git export may fail

CKM-1681

[Internal] Destroy orphaned plugin actors

CKM’s asset management system previously kept a record of orphaned actors from services that no longer exists or were renamed over time. These are not required anymore and should be removed. 83 such actors have been identified and removed.

CKM-1693

[Internal] Improve logging of git push results

The Git push results should be more explictily logged using a log file of its own and separated by errors and info (success).

CKM-1709

[INTERNAL] Improve backend rights management for creating/updating/closing/reactivating/deleting a review round

Improve backend rights management for creating/updating/closing/reactivating/deleting a review round to ensure that even if a user had full access to the backend, a project editor, translation editor, or terminology editor can only create/update review rounds of the appropriate review types and appropriate projects.

Note: This is just an additional precaution. No normal CKM user without elevated rights could do this now, even with full access to the backend. However, a (theoretical) malicious and very tech-savvy project editor could potentially exploit this to create/update/close/reactivate/delete review rounds for other projects.

CKM-1714

[INTERNAL] Extract the User Roles panel as a separate widget

For historical reasons, the user roles tab was implemented directly as part of the User Profile panel.

For better separation and to reduce complexity, this User Roles panel should be refactored/reimplemented as a separate widget that can then be used as one of the tabs in the user profile panel.