Re: Mismatch between CSS and web-platform-tests semantics for reftests

On Tue, Aug 19, 2014 at 3:28 PM, James Graham <james@hoppipolla.co.uk>
wrote:

> On 19/08/14 19:53, Peter Linss wrote:
> > I'm not sure if I'm mis-parsing your explanation or you're
> > misunderstanding the way our references work.
> >
> > To be clear, a test can have multiple reference <links>, if so the
> > test must match one of the references.
> >
> > A reference can <link> to additional references, in this case, the
> > test must match _all_ references in the chain.
> >
> > Yes, the chain can form a tree (but we prefer loops).
>
> Right, this matches my understanding of your model, although I'm not
> sure what you mean by "we prefer loops".
>
> >
> > Both cases are necessary and used. As fantasai pointed out, in some
> > cases there are multiple possible renderings of a feature. In others,
> > it's often possible for a reference to fail in the same way that a
> > test can fail, leading to a false positive. We solve that by having
> > multiple references that use different techniques to render.
> >
> > It's also possible to combine the two cases.
> >
> > Then there are the mismatch references, which can be linked from the
> > test or any of the references. These are also important to detect
> > references that aren't rendering properly.
>
> Yes, these aren't a problem.
>
> > I don't have usage stats of the various cases handy, but I know of a
> > bunch of tests offhand that require each scenario, and I've also
> > recently advised test authors about these features to solve problems
> > they were encountering. This isn't something we can drop or
> > significantly water down.
>
> Can you actually point to some concrete examples? No one has done that
> so far, much less estimated how often these features are required.
>
> Are these features something that any actual implementation is running?
> As far as I can tell from the documentation, Mozilla reftests don't
> support this feature, and I guess from Dirke's response that
> Blink/WebKit reftests don't either. That doesn't cover all possible
> implementations of course.
>

Correct, Blink doesn't support such tests yet.

I doubt WebKit does either, but I haven't checked their tip-of-tree code to
see for sure.

-- Dirk