...
Tests written in the above way can all be run as a full suite in a Continuous Build (aka Devops) environment as is common these days, and it will generate a report in HTML, PDF etc, of exactly the kind described in the certificate section of the Conformance test spec.
SA: I'm pretty sure the tests can also run outside of CB setting and without maven. Furthermore, it may be possible to create standalone native executables for linux and windows platforms using the the recently released Graal VM
Test Data
One of the key issues in the above is having not just test scripts, but test data, generally described by templates and underlying archetypes. To make conformance testing really solid, we need to create numerous (e.g. 10) variant sets that are not necessarily all released to vendors, or else test templates generate on-the-fly by the Conformance test environment.
...