Re: progress on schemas for model testing

On Sun, 2016-07-31 at 12:17 -0500, Shane McCarron wrote:
>   No idea if any clients actually indicate the text
> direction of the annotation.   But I don't think it actually matters
> for interoperability...

Maybe we have different ideas about what "interoperability" means. If
it turns out, for example, that the design means clients can't actually
display mixed Hebrew and French correctly, the design needs to be
fixed.

[...]

> > >  For what it's worth, I note that the textDirection key was added
> > > to the model late in response to feedback from
> > > Internationalization
> > 
> > This makes it more important to test it.

> 
> I disagree.  You are thinking of this data model as if it were some
> sort of an API that needs to be implemented and exercised.  It isn't
> that at all.
> It's just data structures.  And those structures have A LOT of
> optional aspects.

What needs to be tested is that the data structures can be used to
provide the experience that's expected.

>  that there could be annotation use cases requiring this
> > > information
> > > explicitly.  I don't think we have anyone to this point who's
> > > implemented such a use case,
> > 
> > Then it should be removed from the draft, or there should be test
> > cases
> > for it and an implementation found (or made).
> > 
> 
> Hmm.  I appreciate what you are saying here, but why?  What's wrong
> with
> defining a feature in advance of implementation when it is optional?

Why shouldn't we provide large bloated specs that no-one ever
implements fully and that turn out not to make sense if you do try to
implement them? Well, that's one position... :-)

>   The
> conformance tests still exist and, should someone implement it, when
> they
> test their implementation it will not pass if it is broken . 

You can't go past CR with conformance statements that are not tested.

> That everything MANDATORY can be implemented.  At least, that's my
> belief.

No, we test that the entire spec can be implemented interoperably. For
optional features we only test that they can be implemented, so a
single pass is usually accepted, provided that's clearly an expectation
set in the status section of the document when it enters CR. 

This is part of why e.g. some of the CSS specs have sat in CR for a
while - waiting for two implementations of every required feature and,
if they have any, at last one implementation of every optional feature.
Some specs also require that at least one implementation pass every
test, to show that e.g. implementing optional feature A doesn't prevent
optional feature B from working.

Note, in case I wasn't clear - I'm not the gatekeeper here, but rather
someone trying to help the WG and the spec succeed. So what I'm saying
here should be read (I hope) as advice, not as anythig with authority.

Best,

Liam

Received on Sunday, 31 July 2016 18:59:24 UTC