- From: Robin Berjon <robin@w3.org>
- Date: Fri, 23 Aug 2013 15:16:26 +0200
- To: James Graham <james@hoppipolla.co.uk>
- CC: "public-test-infra@w3.org" <public-test-infra@w3.org>
Hi James, On 22/08/2013 18:31 , James Graham wrote: > A modified proposal: Overall, I like this. It seems to address issues with maintainability and developer-friendliness well. Just a few minor quibbles below: > * Any html, xhtml or svg file that links to testharness.js is a > testharness test I wonder if we can brutally simplify this to be for (X)HTML and SVG files that contain the string "testharness.js". It avoids parsing and is likely correct. > * Any html, xhtml or svg file that has a file with the same name but the > suffix -ref before the extension is a reftest file and the corresponding > -ref file is a helper file. I presume that this takes precedence over it linking to testharness (it seems like a more restrictive situation, and we want defined behaviour). > * Any html, xhtml or svg file that contains a link rel=match or link > rel=mismatch is a reftest file. Similarly, I would like to propose that this be specified as: files containing /rel\s*=\s*("|')?\s*(mis)?match\b/. No? Or maybe switch to data-*? > These rules can be overridden by providing an override.manifest file. > Such a file can contain a list of filenames to exclude from the normal > processing above and a list of urls for tests, similar to my previous > proposal. For the sake of consistency, or rather, screw consistency, for the sake of being able to explain and remember such things it would be nice if we stuck to a single terminology. For instance, I presume that your [exclude] section below is listing helper files? If so, it might best be called [helpers] (or whatever else we decide to call "other stuff"). > I am still not sure how to deal with timeouts. One option would be to > put the overall timeout in a meta value rather than in the javascript, > since this will be easier to parse out. For tests where this doesn't > work due to strong constraints on the html, one could use the > override.manifest as above (and also specify the timeout in the js). I > can't say I am thrilled with this idea though. I'm still not entirely convinced that for a big shared project like this in which lots of people with rather different configurations provide input we need anything more for timeout than "regular" and "might take a while". -- Robin Berjon - http://berjon.com/ - @robinberjon
Received on Friday, 23 August 2013 13:16:35 UTC