W3C home > Mailing lists > Public > public-html-testsuite@w3.org > February 2013

Re: Marking non-automated tests

From: Arthur Barstow <art.barstow@nokia.com>
Date: Thu, 07 Feb 2013 07:08:05 -0500
Message-ID: <51139925.6040301@nokia.com>
To: ext Robin Berjon <robin@w3.org>
CC: Kris Krueger <krisk@microsoft.com>, "'public-html-testsuite@w3.org'" <public-html-testsuite@w3.org>
On 2/7/13 3:55 AM, ext Robin Berjon wrote:
> On 06/02/2013 23:36 , Kris Krueger wrote:
>
>> I'd suggest that we have a text file (format/file type is not a
>> concern of mine) that holds this 'meta data' and other metadata (for
>> example test is approved).
>
> I would really rather not. Metadata capture should be designed in such 
> a way that it ensures in as much as possible that it won't go out of 
> date. External authoritative metadata such as in a text file is 
> guaranteed to break.

In general (i.e. not "manual/automated" specific), I agree with Robin 
(to not use an external file).

> Another option is to capture that in file names (if there's ".manual." 
> in the file name, then it's manual).

(I kinda prefer Jame's suggestion to use an appropriately named 
directory for manual tests but I'm mostly indifferent.)

> Finally, we don't need metadata to mark a test as approved. Anything 
> that's in the suite is approved since submissions are in pull requests.

One issue we have with at least one of WebApps' test suites is knowing 
if an approved test is for a specific dated version of the spec or the 
ED. How do you handle that in GH.

-Thanks, AB
Received on Thursday, 7 February 2013 12:08:31 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 7 February 2013 12:08:32 GMT