Re: progress on schemas for model testing

Hi all,

this is a reflection on the whole thread (which I have just read, coming back from vacations) rather than just this specific mail.

In general we should be careful with the analogies with standards like XQuery or CSS. The particularity of the Model spec (and this issue is specific to all specs that define, essentially, a vocabulary) is that there is no observable behaviour associated to the various features, which makes it very different than, say, CSS. To concentrate on the issue of optional terms, such a term does not have an operational effect *as far as the specification is concerned*, because we do not really check the user interface, application behaviour, etc, of annotation systems (we do not specify any of that). In this respect, the issue whether it is impossible to implement that feature or not does not really apply in this case, in my view…

Based on this I pretty much agree with what Shane wrote below. Just one point to be clear: the way I read your answers, Shane, is that we *do* check whether the value of, say, textDirection is correct *if* an implementation implements it, right? I believe that should be the case, although we do not require the presence of that term.

There is, however, an aspect that does come up around 'optional' terms. Ideally, we should document (not necessarily through the formal testing mechanism) that each term, including the 'optional' ones, are *used* in real practice, ie, real implementations. Eg, it would be good to document that, say, Europeana uses (or plans to use) textDirection in their application, thereby proving that it is indeed useful to have that term in the vocabulary as opposed to 'just' have it there without any real usage. That being said, it may be difficult to document that at this point because it is a bit of a chicken-and-egg-problem. But we still may want to try to ask all our implementers whether they have experience or intention to do so.

I hope this helps…

Ivan

> On 31 Jul 2016, at 19:24, Shane McCarron <shane@spec-ops.io> wrote:
> 
> Tim,
> 
> Comments inline:
> 
> On Sat, Jul 30, 2016 at 5:13 PM, Cole, Timothy W <t-cole3@illinois.edu <mailto:t-cole3@illinois.edu>> wrote:
> 
> From: Shane McCarron [shane@spec-ops.io <mailto:shane@spec-ops.io>]
> Sent: Friday, July 29, 2016 09:26
> To: Cole, Timothy W
> Cc: Web Annotation
> Subject: Re: progress on schemas for model testing
> 
> Comments inline:
> 
> On Thu, Jul 28, 2016 at 1:49 AM, Tim Cole <t-cole3@illinois.edu <mailto:t-cole3@illinois.edu>> wrote:
> ...
> 
> 
> One important question before proceeding much further.  - For the purposes of W3C conformance testing, how important is it to report a count of how many times within a single annotation a feature appears, or when multiple bodies or target are involved in an Annotation, do we need to know on which class of body or target a particular feature was used?  For example do we need to count the use of textDirection on embedded Textual Body separately from the use of textDirection on an External Web Resource? Or is it sufficient to simply report that textDirection feature was used on a body or bodies in a given annotation?
> 
> 
> Hmm... I think we need to decide what a "feature" is for CR purposes.  For example, textDirection does not feel like a feature to me.  It is just an attribute of the annotation.  There are lots of attributes, right?  I think that for CR the feature granularity should be at a higher level.  Do annotation clients support the use of bodyValue?  of body? In body, which body types are used?  That's enough.
>  ...
> 
> 
> Looks from the draft minutes like this question / issue was pretty well resolved during Friday's call. If I understand correctly, we generally do not need to test whether SHOULD or MAY keys have been implemented by anyone, at least not for the purposes of exiting CR.  This will drop by half or more the number of schemas we need (though I may keep some of these in the systems to facilitate creating schemas for validating annotations as contrasted to what we have to do for exiting CR). For what it's worth, I note that the textDirection key was added to the model late in response to feedback from Internationalization that there could be annotation use cases requiring this information explicitly.  I don't think we have anyone to this point who's implemented such a use case, so probably just as well if we don't look for the textDirection key, as long as not doing so doesn't preclude leaving the key in the model and as long as not doing so will still allow us to exit CR.
> 
> We test for it.  It just isn't a failure if it isn't present.
> 
> 
> Do need clarification on two points, however:
> 
> A. Given that we are not tracking if keys like textDirection have been implemented, do we even need to check (again, just for the purposes of documenting feature implementation in order to exit CR) that if implemented, the value of textDirection if present is correct? For validation, obviously this would be a good thing to do and the schema in this instance is trivial, but the east of checking the validity of textDirection values is atypical; it's actually not practical to use JSON schema to check validity (both MUST and SHOULD) of many key values (e.g., not practical to verify that all values of the language key conform to bcp47). So if in order  to satisfy CR exit requirements, we don't need to include any test relating to textDirection (or its value) when implemented, so much the better.
> 
> Yes - we definitely should be checking this.\
> 
> 
> B. Assuming that class of body / class of target implemented is a feature, do we need to differentiate when (for example) a SpecificResource is used as a body vs. when it is used as a target? JSON schema patternProperty would allow us to check for implementation of SpecificResource without having to differentiate whether it appeared as a body or a target.
> 
> Yes.  I think there should be a test for SpecificResource as a target and for SpecificResource as a body.  The test is whether the feature is supported, and those are two very different features.
> 
> 
> Depending on answer to these 2 questions, our model testing may reduce to:
> 
> 3.1-Annotations:
>    test for @context, type, id, target, not both body and bodyValue
> 
> 3.2-BodiesTargets (together rather than separately):
>    test to determine classes of bodies and targets described in the model are implemented, specifically which of External Web Resources, String Body (bodyValue), Embedded Textual Body (body only), Choice, Composite, List, Independents, SpecificResource are used as a body or target. For the last 4, we presumably still want to test which Body/Target classes are then used to populate each items array (Choice, ..., Independents) and the source key of each SpecificResource.
> 
> 3.3-OtherProperties:
>       no tests, unless we want to test created, generated, modified (all MUST be date-time) and possibly also for motivation (add to Annotation test, a SHOULD key) or purpose (add to BodiesTargets - MAY key). Note we provide a starting point list of recommended values for these last 2 keys and we could test to see which from this ontology is used.
> 
> 4.-SpecificResources:
>     test to determine which classes of Selectors and/or, States are used. Implementation of Styles, Rendering Software and Scope could be tracked too, but these really have no MUSTS and or themselves SHOULD or MAY. .
> 
> 5- CollectionsPages:
>     test @context, type, id, first, items, next and possibly the correctness of label, total, last, prev, startIndex when present.
> 
> Does this inventory sound about right?
> 
> 
> I suspect it is slightly more complex - see above.  BUt yes, that level of granularity appears correct to me.
> 
> -Tim Cole
> 
> --
> Shane McCarron
> Projects Manager, Spec-Ops
> 
> 
> 
> 
> --
> Shane McCarron
> Projects Manager, Spec-Ops


----
Ivan Herman, W3C
Digital Publishing Lead
Home: http://www.w3.org/People/Ivan/
mobile: +31-641044153
ORCID ID: http://orcid.org/0000-0003-0782-2704