Re: marking tests that require user interaction?

On May 23, 2013, at 5:17 AM, Robin Berjon wrote:

> On 23/05/2013 13:36 , James Graham wrote:
>> True, but not really the answer to the current question.
>> 
>> As discussed on irc, the de-facto standard is to use a MANIFEST file with
>> 
>> manual foo.html
>> 
>> to indicate that a test is manual.
>> 
>> That works with Mozilla's infrastructure at least and since they seem to
>> be one of very few people actively importing W3C tests it makes sense to
>> use a system that works for them.
> 
> My alternative proposal was to have manual tests called foo.manual.html, to avoid manifests going out of sync with reality. From there it's easy to generate a manifest automatically if needed.
> 
> But I'm happy to go with whatever is done by whoever does the work.
> 
> -- 
> Robin Berjon - http://berjon.com/ - @robinberjon
> 

The CSSWG uses a metadata flag, 'interact':
http://wiki.csswg.org/test/format#requirement-flags

We don't favor manifest files in the test source tree as they can easily get out of sync with the tests. We do have a build process that generates manifests for external test running systems using the test's internal metadata (I'm actually in the process of making that build process generic now).

Peter

Received on Thursday, 23 May 2013 18:24:18 UTC