Re: Requirements flags in reference files

On Jun 26, 2013, at 9:00 PM, Gérard Talbot wrote:

> 
> Le Mer 26 juin 2013 21:25, Peter Linss a écrit :
>> It's perfectly valid for a reference file to have flags,
> 
> 
> Well, then, that is news to me, Peter. Big surprise here!
> 
> "
> Unlike the format for the test file, there is no metadata except for the
> author credits and optional reference links.
> "
> http://wiki.csswg.org/test/reftest#the-reftest-reference-file
> 
> and there is an example given... and no metadata flags in the simplified
> reference format...
> 
> I could also swear we once discussed this issue in an email...
> 
> 
>> and they should
>> be present as needed,
> 
> !!

It's not all that crucial at the moment, as the test harness isn't paying attention to them (yet). The test harness *should* be using the union of all the flags in the test and the reference files.

The flags that affect the build system (which for references should only be 'HTMLonly' and 'nonHTML') do apply (but I suspect we don't have any references that are format specific currently).

> 
>> but they should only reflect requirements of the
>> reference, not the test. e.g. if the test uses the ahem font, but the
>> reference doesn't, then the reference should not have the 'ahem' flag.
>> 
>> The flags are used by the test harness and the build code.
>> 
> 
> I have created dozens of reference files that were using the Ahem font
> and never declared the ahem flag.

If they're all used for tests that also have the 'ahem' flag, then it's not really a big deal as the tests will cover the requirement, but it's a good idea to have them for safety.

> 
> Gérard
> 
>> The build code uses some flags like 'HTMLonly' and 'nonHTML' to control
>> format conversions, and 'asis' to avoid re-serialization.
>> 
>> The harness will list conditions triggered by the flags, and tests can
>> be filtered by flag (say, skip all the 'ahem' tests if you don't have it
>> installed, or all the 'http' tests if you're running locally).
>> 
>> The harness also uses 'may' and 'should' to determine which tests are
>> optional for computing CR exit requirements and generating
>> implementation reports.
>> 
>> Other flags are for future use, like the 'animated' or 'interact' flags
>> will tell an automatic reference comparator to not bother.
>> 
>> Also note that some flags will get injected automatically by the build
>> code, such as 'reftest' when it detects reference links.
>> 
>> Peter
>> 
>> On Jun 26, 2013, at 4:45 PM, Rebecca Hauck wrote:
>> 
>>> Hey Peter & Gérard,
>>> 
>>> I've seen Gérard remove these flags in the past [1], but the
>>> guidelines [2] are somewhat ambiguous. Should a ref file not have
>>> flags? For cases where the test and the reference have the same flag
>>> (for example, ahem), it makes sense that you needn't duplicate the
>>> info in the ref file as its bound to the test.  What about cases where
>>> the ref file has a flag that the test file does not? An example off
>>> the top of my head is maybe using SVG to render something equivalent
>>> that you're testing in CSS.
>>> 
>>> And more generally, how exactly are these flags used?  I saw some
>>> confusion about flags at TestTWF Tokyo as well that raises this
>>> question for me.
>>> 
>>> (cc'ing my colleague Bem, who's not subscribed to this list, but who's
>>> encountering this situation)
>>> 
>>> Thanks,
>>> -Rebecca
>>> 
>>> [1] http://hg.csswg.org/test/rev/ed3f96bded71
>>> [2] http://wiki.csswg.org/test/format#requirement-flags
>> 
>> 
> 
> 
> -- 
> Contributions to the CSS 2.1 test suite:
> http://www.gtalbot.org/BrowserBugsSection/css21testsuite/
> 
> CSS 2.1 Test suite RC6, March 23rd 2011:
> http://test.csswg.org/suites/css2.1/20110323/html4/toc.html
> 
> CSS 2.1 test suite harness:
> http://test.csswg.org/harness/
> 
> Contributing to to CSS 2.1 test suite:
> http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html
>

Received on Thursday, 27 June 2013 05:56:31 UTC