progress on schemas for model testing

Following up on last 2 Working Group calls, we've re-organized and consolidated the schemas in the https://github.com/w3c/web-annotation-tests repository. We do not yet have repository fully populated with all required schema, but we have made significant progress with the new approach and will continue if the consensus of the WG is that we're now on the right track.

 

The /definitions folder now contains 6 JSON files, each of which contains multiple sub-schemas that can be referenced directly by other schemas to facilitate testing and validation. Most of these files are not yet fully populated, and we may need additional files, but consolidating in a smaller number of files will facilitate management of these definitional schemas and will acilitate loading them as reference schemas. The files roughly align with sections of the model spec:

 

Annotations.json - annotation-specific properties, checks and constraints like @context and "type": "Annotation", Section 3.1

 

bodyTarget.json - properties, checks and constraints related to targets and/or bodies, Section 3.2.1 - 3.2.6

 

choiceSet.json - properties, checks and constraints related to choice, composite, list, independents,, Section 3.2.7 - 3.2.8

 

otherProperties.json - life cycle and other properties, checks and constraints used with annotations, bodies and targets , Section 3.3

 

specificResource.json - properties, checks and constraints related to Specific Resources, Section 4.

 

id.json - checks to validate id and strings of format uri

 

Other folders in the repository contain higher level schemas that can be referenced from test scripts or that can be useful for validation. For example,  /annotations/3.1-annotationMustKeys.json contains schemas for reference by test scripts for the MUST requirements for Annotations - i.e., @context, id that is URI, type, only one of body and bodyValue, target. This schema can also be used independent of the WPT test tools as a JSON Schema way to validate presence and correctness of mandatory Annotation keys.   

 

/bodiesTargets/3.2.1-6-BodyResources.json and /bodiesTargets/3.2.1-6-TargetResources.json contain schemas for detecting and validating (to an extent at least) optional feature implementation, differentiating between different kinds of bodies and targets and validating value correctness of mostly body/target optional key values.  Again these 2 schemas, assuming the local availability of validating parsers like ajv-cli, can be used to locally validate correctness of optional keys - i.e., giving a valid result if feature is present and correct, or is not present at all, but giving an invalid result if the feature is present but is incorrect (e.g., if an object rather than a string or if value is not consistent with enumerated list of values). There are also some schema (with incorrect  expectedResult values I notice now) that can be used to report that a specific feature has been implemented..

 

Please note that these schema are unfinished and not fully checked. You'll find errors but it’s a good start and hopefully makes a little more sense than what we had 2 weeks ago..

 

One important question before proceeding much further.  - For the purposes of W3C conformance testing, how important is it to report a count of how many times within a single annotation a feature appears, or when multiple bodies or target are involved in an Annotation, do we need to know on which class of body or target a particular feature was used?  For example do we need to count the use of textDirection on embedded Textual Body separately from the use of textDirection on an External Web Resource? Or is it sufficient to simply report that textDirection feature was used on a body or bodies in a given annotation?

 

Otherwise please comment and/or contribute. You'll note that full naming convention is not fully in place yet. Test scripts still need to be organized and written.  Logic seems a bit non-intuitive, which may be the nature of how we're trying to use json schema; but it also may be just bad schema writing.  

 

Thanks,

 

Tim Cole