W3C home > Mailing lists > Public > public-test-infra@w3.org > April to June 2012

Re: Repository layout

From: Linss, Peter <peter.linss@hp.com>
Date: Fri, 1 Jun 2012 14:52:00 +0000
To: James Graham <jgraham@opera.com>
CC: "<public-test-infra@w3.org>" <public-test-infra@w3.org>
Message-ID: <43EAC896-AEF6-4DD8-A728-89552AE4B56A@hp.com>

On Jun 1, 2012, at 7:03 AM, James Graham wrote:

> On 06/01/2012 03:58 PM, Linss, Peter wrote:
> 
>> There's a fundamental problem here, while I understand that most of
>> the suites don't (yet) use a build step, and many won't need one (or
>> may not be able to use one), there are some test suites where a build
>> step is _mandatory_, like the CSS suites. And that's not going to
>> change, if anything we're getting more dependent on the build process
>> over time.
>> 
>> If you can't incorporate a build step into your workflow, then there
>> are a _lot_ of tests that will be unavailable to you, like around
>> 11,000 CSS tests (and that number is growing rapidly). (Actually if
>> you consider each format the tests are available in, that's more like
>> 33,000 tests.)
> 
> We *can* incorporate that step of course, but we would *much prefer* not to. It makes updating the tests a significantly more complex undertaking than it would otherwise be. This means we are less likely to run an up-to-date copy of the tests and as a result interoperability will suffer.


Ok, so let me try to sum up then (since this thread has gone off in a few different directions).

* None of the tools require any particular repository layout (those that care about it are configurable). 
* Each group is free to organize their repositories as they see fit. They should evaluate their needs and organize to suit. For simple suites a flat, stable structure is preferred.
* The approved/submitted separation is a carry over from before the CSS WG had a management tool to maintain test status. (Even though we have such a tool now, we do still find it useful to organize our tests that way for the broad strokes, it helps people working on tests offline, but the placement of the file in the repository is not the final determination of its status).
* Using the build tools separates the organization of the repository from the built test suite.
* Consumers of test suites may prefer not to require a build step, but they will have to deal with it anyway for some suites at least.
* When the build tools are ready for general use, they should be used to generate manifests for consumption of the test suites with the best possible metadata. (I'm working this.)
* Something else that I'm forgetting…

Peter
Received on Friday, 1 June 2012 14:52:58 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 1 June 2012 14:52:59 GMT