W3C home > Mailing lists > Public > public-test-infra@w3.org > July to September 2014

Re: Mismatch between CSS and web-platform-tests semantics for reftests

From: Dirk Pranke <dpranke@chromium.org>
Date: Tue, 19 Aug 2014 16:41:19 -0700
Message-ID: <CAEoffTDDELDyg1vgXRZ0GMhWtYuBQvBi+vatTa26AzTMgC8Cwg@mail.gmail.com>
To: James Graham <james@hoppipolla.co.uk>
Cc: public-test-infra <public-test-infra@w3.org>
On Tue, Aug 19, 2014 at 3:28 PM, James Graham <james@hoppipolla.co.uk>

> On 19/08/14 19:53, Peter Linss wrote:
> > I'm not sure if I'm mis-parsing your explanation or you're
> > misunderstanding the way our references work.
> >
> > To be clear, a test can have multiple reference <links>, if so the
> > test must match one of the references.
> >
> > A reference can <link> to additional references, in this case, the
> > test must match _all_ references in the chain.
> >
> > Yes, the chain can form a tree (but we prefer loops).
> Right, this matches my understanding of your model, although I'm not
> sure what you mean by "we prefer loops".
> >
> > Both cases are necessary and used. As fantasai pointed out, in some
> > cases there are multiple possible renderings of a feature. In others,
> > it's often possible for a reference to fail in the same way that a
> > test can fail, leading to a false positive. We solve that by having
> > multiple references that use different techniques to render.
> >
> > It's also possible to combine the two cases.
> >
> > Then there are the mismatch references, which can be linked from the
> > test or any of the references. These are also important to detect
> > references that aren't rendering properly.
> Yes, these aren't a problem.
> > I don't have usage stats of the various cases handy, but I know of a
> > bunch of tests offhand that require each scenario, and I've also
> > recently advised test authors about these features to solve problems
> > they were encountering. This isn't something we can drop or
> > significantly water down.
> Can you actually point to some concrete examples? No one has done that
> so far, much less estimated how often these features are required.
> Are these features something that any actual implementation is running?
> As far as I can tell from the documentation, Mozilla reftests don't
> support this feature, and I guess from Dirke's response that
> Blink/WebKit reftests don't either. That doesn't cover all possible
> implementations of course.

Correct, Blink doesn't support such tests yet.

I doubt WebKit does either, but I haven't checked their tip-of-tree code to
see for sure.

-- Dirk
Received on Tuesday, 19 August 2014 23:42:06 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:11 UTC