W3C home > Mailing lists > Public > public-test-infra@w3.org > April to June 2012

Re: Repository layout

From: Linss, Peter <peter.linss@hp.com>
Date: Wed, 30 May 2012 17:34:36 +0000
To: James Graham <jgraham@opera.com>
CC: "<public-test-infra@w3.org>" <public-test-infra@w3.org>
Message-ID: <7740010B-9C94-41C8-B532-84FBDFA3E071@hp.com>
On May 30, 2012, at 4:53 AM, James Graham wrote:

> For many test repositories we are using a submitted/approved directory structure. This is not working well for several reasons:
> 
> * There are typically far more useful tests in the submitted directories than the approved directories. This is due to a general lack of time/interest in reviewing tests (I am just as guilty as anyone). I doubt this situation will change.
> 
> * It makes it difficult for us to import the tests. Because of the way we test it is very helpful if the paths to tests remain constant (cool URIs and all that). Moving the tests around is a severe inconvenience as we have to update our metadata with the new paths.
> 
> I suggest we go with a single-level directory structure. If people want to keep metadata about which tests have been reviewed that should not be encoded in the filesystem heirachy. I doubt we will do any better than assuming that tests are good and that implementors will file bugs when they find a bad test.
> 


The submitted/approved split is a carryover from the CSS test repository before we had a test tracking tool. Even though we do have a tool now, we still find it somewhat useful to have a distinction between 'blessed' tests and those that are being worked on and haven't had any real kind of review.

The expectation when other tools import the tests is that they not import directly from the source tree, but from the build output. The CSS test suites have build tools that gather all the tests in the various source directories and copy the proper tests into per-suite output directories. The build tools also generate manifest files for other systems to import the test data (and metadata) as well as human readable index files.

I'm working on making the CSS build tools more generic so they can be used with other test suites. Once that's done, if you use the build output then the layout of the source repository is irrelevant to your tools.

Peter
Received on Wednesday, 30 May 2012 17:36:10 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 30 May 2012 17:36:11 GMT