RE: Test Case Template/Meta Data

Quoting Kris Krueger <krisk@microsoft.com>:

> How about we agree on doing this...
>
> Some test cases (the example we have been using) are too complex
> to link back to a very specific part of the HTML5 spec.
>
> Though we could at least categorize this test case as a complex
> parser test and link it back to '9.2 Parsing HTML documents'.

Yes, we should undoubtedly categorise tests at a high level. I suggest  
simply using the filesystem structure to do this, so we would have  
different folders for say parsing, canvas, media elements, and so on.  
This would basically follow the high level structure of the spec.  
Within each folder we could have more structure probably initially  
some subdivison depending on the type of the test (some tests might  
use javascript, some might be reftests, etc.) and possibly further  
levels of nesting to account for finer grained structure in the  
section in question.

> A template could be used with an xml file checked into CVS. So that
> it would be possible to generate a test page similar to below using
> script.

On the subject of CVS (but somewhat at a tangent to the rest of this  
thread), I have a strong preference to avoid CVS and use something  
like mercurial if at all possible. I know that some of the other  
testing work inside the W3C uses mercurial for version control so  
hopefully it's possible for us too.

The main reason I have for preferring a DVCS is that the use (and to a  
lesser extent, development) of the testsuite is essentially  
distributed. For example Opera will want to use the tests for our  
internal regression testing. This means taking the testsuite, possibly  
making some modifications so that it talks to our regression detection  
system, and keeping the modified testsuite in our local VCS. With a  
DVCS changes to the master copy can easilly be pulled and applied to  
the local clone. With a centralized VCS this is not quite so trivial.

I also generally find mercurial nicer to work with than CVS :)

Received on Friday, 20 November 2009 23:35:56 UTC