W3C home > Mailing lists > Public > public-css-testsuite@w3.org > June 2013

Re: Requirements flags in reference files

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Thu, 27 Jun 2013 00:00:27 -0400
Message-ID: <7f9027146ca3da589fd7c1fe89bfc058.squirrel@ed-sh-cp3.entirelydigital.com>
To: "Peter Linss" <peter.linss@hp.com>
Cc: "Rebecca Hauck" <rhauck@adobe.com>, "Bem Jones-Bey" <bjonesbe@adobe.com>, "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>

Le Mer 26 juin 2013 21:25, Peter Linss a écrit :
> It's perfectly valid for a reference file to have flags,

Well, then, that is news to me, Peter. Big surprise here!

Unlike the format for the test file, there is no metadata except for the
author credits and optional reference links.

and there is an example given... and no metadata flags in the simplified
reference format...

I could also swear we once discussed this issue in an email...

> and they should
> be present as needed,


> but they should only reflect requirements of the
> reference, not the test. e.g. if the test uses the ahem font, but the
> reference doesn't, then the reference should not have the 'ahem' flag.
> The flags are used by the test harness and the build code.

I have created dozens of reference files that were using the Ahem font
and never declared the ahem flag.


> The build code uses some flags like 'HTMLonly' and 'nonHTML' to control
> format conversions, and 'asis' to avoid re-serialization.
> The harness will list conditions triggered by the flags, and tests can
> be filtered by flag (say, skip all the 'ahem' tests if you don't have it
> installed, or all the 'http' tests if you're running locally).
> The harness also uses 'may' and 'should' to determine which tests are
> optional for computing CR exit requirements and generating
> implementation reports.
> Other flags are for future use, like the 'animated' or 'interact' flags
> will tell an automatic reference comparator to not bother.
> Also note that some flags will get injected automatically by the build
> code, such as 'reftest' when it detects reference links.
> Peter
> On Jun 26, 2013, at 4:45 PM, Rebecca Hauck wrote:
>> Hey Peter & Gérard,
>> I've seen Gérard remove these flags in the past [1], but the
>> guidelines [2] are somewhat ambiguous. Should a ref file not have
>> flags? For cases where the test and the reference have the same flag
>> (for example, ahem), it makes sense that you needn't duplicate the
>> info in the ref file as its bound to the test.  What about cases where
>> the ref file has a flag that the test file does not? An example off
>> the top of my head is maybe using SVG to render something equivalent
>> that you're testing in CSS.
>> And more generally, how exactly are these flags used?  I saw some
>> confusion about flags at TestTWF Tokyo as well that raises this
>> question for me.
>> (cc'ing my colleague Bem, who's not subscribed to this list, but who's
>> encountering this situation)
>> Thanks,
>> -Rebecca
>> [1] http://hg.csswg.org/test/rev/ed3f96bded71
>> [2] http://wiki.csswg.org/test/format#requirement-flags

Contributions to the CSS 2.1 test suite:

CSS 2.1 Test suite RC6, March 23rd 2011:

CSS 2.1 test suite harness:

Contributing to to CSS 2.1 test suite:
Received on Thursday, 27 June 2013 04:00:55 UTC

This archive was generated by hypermail 2.4.0 : Friday, 20 January 2023 19:58:19 UTC