Re: Mismatch between CSS and web-platform-tests semantics for reftests

I'm not sure if I'm mis-parsing your explanation or you're misunderstanding the way our references work. 

To be clear, a test can have multiple reference <links>, if so the test must match one of the references.

A reference can <link> to additional references, in this case, the test must match _all_ references in the chain.

Yes, the chain can form a tree (but we prefer loops). 

Both cases are necessary and used. As fantasai pointed out, in some cases there are multiple possible renderings of a feature. In others, it's often possible for a reference to fail in the same way that a test can fail, leading to a false positive. We solve that by having multiple references that use different techniques to render.

It's also possible to combine the two cases.

Then there are the mismatch references, which can be linked from the test or any of the references. These are also important to detect references that aren't rendering properly.

I don't have usage stats of the various cases handy, but I know of a bunch of tests offhand that require each scenario, and I've also recently advised test authors about these features to solve problems they were encountering. This isn't something we can drop or significantly water down.

On Aug 19, 2014, at 3:01 AM, James Graham <james@hoppipolla.co.uk> wrote:

> web-platform-tests has a rather simple set of semantics for reftests;
> each test is identified by a single (test, ref, type) tuple. Multiple
> <link> elements in the same test file result in multiple tests with
> different references.
> 
> By contrast the testthewebforward documentation suggests that CSS has a
> rather more complex semantic; each test is of the form (test, ref_tree),
> where ref_tree is a tree of (file, type) nodes and breadth in the tree
> indicates multiple options and depth in the tree indicates multiple
> requirements, so the overall test passes if all the comparisons down to
> some leaf node pass. In concrete terms multiple <link> elements in the
> same document represent multiple options, and each reference can <link>
> to further references.
> 
> Do we know how necessary the more complex setup is? In particular I
> think that the main distinguishing feature is the ability to specify
> multiple references of which *any* may match (since the all must match
> semantic is just a way of collapsing N tests down to a single test). Is
> there a list of all tests that are using this feature?
>