Re: Marking non-automated tests

On 07/02/2013 10:24 , James Graham wrote:
> On 02/07/2013 09:55 AM, Robin Berjon wrote:
>> I would really rather not. Metadata capture should be designed in such a
>> way that it ensures in as much as possible that it won't go out of date.
>> External authoritative metadata such as in a text file is guaranteed to
>> break. That's why I proposed inlining it (in the most lightweight manner
>> I could think of).
>
> Inline metadata has the disadvantage that it is hard to extract
> (requires a HTML parser in this case)

Dirty secret: I was thinking that:

   fs.readFileSync("test.html", "utf8).match(/\ddata-manual\b/)

would do the trick :)

> and can affect the test itself.

It could, I guess, if the implementation does something really weird (or 
you're doing a manual test that enumerates stuff on <html>, which I 
would likely frown upon).

>> Another option is to capture that in file names (if there's ".manual."
>> in the file name, then it's manual).
>
> That option works for me, as long as "javascript" is considered the
> default.

Well of course.

I can live with using the file name (a convention that we can extend to 
reftests as well). I don't have a strong preference between that and 
inlining but since you do I'll go with the flow. Anyone disagrees?

> Alternatively I am happy with a specific sub-directory for non
> automated tests

We use directories to map to spec sections — I'd rather keep it that way.

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Thursday, 7 February 2013 10:35:06 UTC