W3C home > Mailing lists > Public > public-html-testsuite@w3.org > February 2013

Re: Marking non-automated tests

From: Robin Berjon <robin@w3.org>
Date: Thu, 07 Feb 2013 13:13:59 +0100
Message-ID: <51139A87.3040200@w3.org>
To: Arthur Barstow <art.barstow@nokia.com>
CC: Kris Krueger <krisk@microsoft.com>, "'public-html-testsuite@w3.org'" <public-html-testsuite@w3.org>
On 07/02/2013 13:08 , Arthur Barstow wrote:
> On 2/7/13 3:55 AM, ext Robin Berjon wrote:
>> Another option is to capture that in file names (if there's ".manual."
>> in the file name, then it's manual).
>
> (I kinda prefer Jame's suggestion to use an appropriately named
> directory for manual tests but I'm mostly indifferent.)

The problem with using a specific directory is that we then lose the 
mapping from the test to the specification section (an important piece 
of information that we're using to assess coverage).

>> Finally, we don't need metadata to mark a test as approved. Anything
>> that's in the suite is approved since submissions are in pull requests.
>
> One issue we have with at least one of WebApps' test suites is knowing
> if an approved test is for a specific dated version of the spec or the
> ED. How do you handle that in GH.

The way we're handling that is that we have the master branch for 
"latest whatever" (ED) and a CR branch that maps to the latest snapshot. 
I *think* it would be a bad idea to complexify this to the point where 
we might need to map to multiple previous snapshots so I reckon in your 
case you can do the same and use CR for snapshots (even if they aren't 
in CR).

-- 
Robin Berjon - http://berjon.com/ - @robinberjon
Received on Thursday, 7 February 2013 12:14:10 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 7 February 2013 12:14:10 GMT