Re: Policy for reftests that require features from other specs?

On 05/21/2014 06:46 PM, Daniel Holbert wrote:
> While I was clicking through the flexbox tests liked off of the ED[1], I
> noticed that some of the tests served by the testsuite (at the moment)
> are really tests for interactions between flexbox & regions.[2]
> Obviously, these tests fail in UAs that don't implement regions.
>
> Is there any sort of policy on where to put tests like these ones, that
> depend on features from several in-development mostly-unrelated CSS
> specs?  On the one hand, it's absolutely worthwhile to test the
> interaction of features. But given that we're grouping testsuites in a
> per-spec way right now, it seems odd that we'd implicitly mandate that
> UAs implement an in-development "Spec B" before they can pass the
> testsuite for a mature very-close-to-CR "Spec A". (except perhaps in
> cases where there was a tight coupling between the two specs)
>
> Thoughts? Is this already laid out somewhere?  Am I reading too much
> into what the testsuites mean?

The policy is definitely that we accept such tests,
and that a UA is only required to pass them if it
claims to implement both modules.

However, it does seem the test results UI on the specs
isn't making this very clear atm.

Does anyone have a suggestion for improvement here?

~fantasai

> [1] http://test.csswg.org/harness/test/css-flexbox-1_dev/
>
> [2] e.g.:
> http://test.csswg.org/harness/test/css-flexbox-1_dev/autoheight-flexbox-001/
> (through 004)
> http://test.csswg.org/harness/test/css-flexbox-1_dev/autoheight-regions-in-autoheight-flexbox-001/
> (through 004)
> http://test.csswg.org/harness/test/css-flexbox-1_dev/autoheight-regions-in-fixed-sized-flexbox-001/
> (through 008)

Received on Wednesday, 2 July 2014 08:21:35 UTC