W3C home > Mailing lists > Public > public-css-testsuite@w3.org > June 2013

Re: Requirements flags in reference files

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Thu, 27 Jun 2013 00:00:27 -0400
Message-ID: <7f9027146ca3da589fd7c1fe89bfc058.squirrel@ed-sh-cp3.entirelydigital.com>
To: "Peter Linss" <peter.linss@hp.com>
Cc: "Rebecca Hauck" <rhauck@adobe.com>, "Bem Jones-Bey" <bjonesbe@adobe.com>, "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>

Le Mer 26 juin 2013 21:25, Peter Linss a écrit :
> It's perfectly valid for a reference file to have flags,


Well, then, that is news to me, Peter. Big surprise here!

"
Unlike the format for the test file, there is no metadata except for the
author credits and optional reference links.
"
http://wiki.csswg.org/test/reftest#the-reftest-reference-file

and there is an example given... and no metadata flags in the simplified
reference format...

I could also swear we once discussed this issue in an email...


> and they should
> be present as needed,

!!

> but they should only reflect requirements of the
> reference, not the test. e.g. if the test uses the ahem font, but the
> reference doesn't, then the reference should not have the 'ahem' flag.
>
> The flags are used by the test harness and the build code.
>

I have created dozens of reference files that were using the Ahem font
and never declared the ahem flag.

Gérard

> The build code uses some flags like 'HTMLonly' and 'nonHTML' to control
> format conversions, and 'asis' to avoid re-serialization.
>
> The harness will list conditions triggered by the flags, and tests can
> be filtered by flag (say, skip all the 'ahem' tests if you don't have it
> installed, or all the 'http' tests if you're running locally).
>
> The harness also uses 'may' and 'should' to determine which tests are
> optional for computing CR exit requirements and generating
> implementation reports.
>
> Other flags are for future use, like the 'animated' or 'interact' flags
> will tell an automatic reference comparator to not bother.
>
> Also note that some flags will get injected automatically by the build
> code, such as 'reftest' when it detects reference links.
>
> Peter
>
> On Jun 26, 2013, at 4:45 PM, Rebecca Hauck wrote:
>
>> Hey Peter & Gérard,
>>
>> I've seen Gérard remove these flags in the past [1], but the
>> guidelines [2] are somewhat ambiguous. Should a ref file not have
>> flags? For cases where the test and the reference have the same flag
>> (for example, ahem), it makes sense that you needn't duplicate the
>> info in the ref file as its bound to the test.  What about cases where
>> the ref file has a flag that the test file does not? An example off
>> the top of my head is maybe using SVG to render something equivalent
>> that you're testing in CSS.
>>
>> And more generally, how exactly are these flags used?  I saw some
>> confusion about flags at TestTWF Tokyo as well that raises this
>> question for me.
>>
>> (cc'ing my colleague Bem, who's not subscribed to this list, but who's
>> encountering this situation)
>>
>> Thanks,
>> -Rebecca
>>
>> [1] http://hg.csswg.org/test/rev/ed3f96bded71
>> [2] http://wiki.csswg.org/test/format#requirement-flags
>
>


-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 Test suite RC6, March 23rd 2011:
http://test.csswg.org/suites/css2.1/20110323/html4/toc.html

CSS 2.1 test suite harness:
http://test.csswg.org/harness/

Contributing to to CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html
Received on Thursday, 27 June 2013 04:00:55 UTC

This archive was generated by hypermail 2.4.0 : Friday, 20 January 2023 19:58:19 UTC