Re: Mismatch between CSS and web-platform-tests semantics for reftests

On 19/08/14 19:53, Peter Linss wrote:
> I'm not sure if I'm mis-parsing your explanation or you're
> misunderstanding the way our references work.
> 
> To be clear, a test can have multiple reference <links>, if so the
> test must match one of the references.
> 
> A reference can <link> to additional references, in this case, the
> test must match _all_ references in the chain.
> 
> Yes, the chain can form a tree (but we prefer loops).

Right, this matches my understanding of your model, although I'm not
sure what you mean by "we prefer loops".

> 
> Both cases are necessary and used. As fantasai pointed out, in some
> cases there are multiple possible renderings of a feature. In others,
> it's often possible for a reference to fail in the same way that a
> test can fail, leading to a false positive. We solve that by having
> multiple references that use different techniques to render.
> 
> It's also possible to combine the two cases.
> 
> Then there are the mismatch references, which can be linked from the
> test or any of the references. These are also important to detect
> references that aren't rendering properly.

Yes, these aren't a problem.

> I don't have usage stats of the various cases handy, but I know of a
> bunch of tests offhand that require each scenario, and I've also
> recently advised test authors about these features to solve problems
> they were encountering. This isn't something we can drop or
> significantly water down.

Can you actually point to some concrete examples? No one has done that
so far, much less estimated how often these features are required.

Are these features something that any actual implementation is running?
As far as I can tell from the documentation, Mozilla reftests don't
support this feature, and I guess from Dirke's response that
Blink/WebKit reftests don't either. That doesn't cover all possible
implementations of course.

Received on Tuesday, 19 August 2014 22:28:51 UTC