Re: Knowing which tests are in the repository

Hi James,

On 22/08/2013 18:31 , James Graham wrote:
> A modified proposal:

Overall, I like this. It seems to address issues with maintainability 
and developer-friendliness well. Just a few minor quibbles below:

> * Any html, xhtml or svg file that links to testharness.js is a
> testharness test

I wonder if we can brutally simplify this to be for (X)HTML and SVG 
files that contain the string "testharness.js". It avoids parsing and is 
likely correct.

> * Any html, xhtml or svg file that has a file with the same name but the
> suffix -ref before the extension is a reftest file and the corresponding
> -ref file is a helper file.

I presume that this takes precedence over it linking to testharness (it 
seems like a more restrictive situation, and we want defined behaviour).

> * Any html, xhtml or svg file that contains a link rel=match or link
> rel=mismatch is a reftest file.

Similarly, I would like to propose that this be specified as: files 
containing /rel\s*=\s*("|')?\s*(mis)?match\b/. No? Or maybe switch to 
data-*?

> These rules can be overridden by providing an override.manifest file.
> Such a file can contain a list of filenames to exclude from the normal
> processing above and a list of urls for tests, similar to my previous
> proposal.

For the sake of consistency, or rather, screw consistency, for the sake 
of being able to explain and remember such things it would be nice if we 
stuck to a single terminology. For instance, I presume that your 
[exclude] section below is listing helper files? If so, it might best be 
called [helpers] (or whatever else we decide to call "other stuff").

> I am still not sure how to deal with timeouts. One option would be to
> put the overall timeout in a meta value rather than in the javascript,
> since this will be easier to parse out. For tests where this doesn't
> work due to strong constraints on the html, one could use the
> override.manifest as above (and also specify the timeout in the js). I
> can't say I am thrilled with this idea though.

I'm still not entirely convinced that for a big shared project like this 
in which lots of people with rather different configurations provide 
input we need anything more for timeout than "regular" and "might take a 
while".

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Friday, 23 August 2013 13:16:35 UTC