W3C home > Mailing lists > Public > public-test-infra@w3.org > April to June 2012

Re: Repository layout

From: Robin Berjon <robin@berjon.com>
Date: Mon, 4 Jun 2012 17:46:50 +0200
Cc: public-test-infra@w3.org
Message-Id: <7770BD2D-C390-49EB-B94D-0CA4F69D52FC@berjon.com>
To: James Graham <jgraham@opera.com>
On Jun 1, 2012, at 14:22 , James Graham wrote:
> On 06/01/2012 02:01 PM, Robin Berjon wrote:
>> On May 31, 2012, at 16:15 , James Graham wrote:
>>>> Overall I agree that the approval step is a bureaucratic speed bump
>>>> that is not being helpful. I think that we should move to a
>>>> commit-then-review model in which people who have an interest in
>>>> passing a test suite can file bugs against broken tests. Ideally, we
>>>> would make flagging broken tests easy  I'm thinking about ways of
>>>> doing that in the framework (suggestions welcome  I wonder if I
>>>> could just add a flag to "reported as broken" test cases).
>>> Make it possible to link to bug reports?
>> Are you thinking of something like<link rel='bug' href='URL of bug report'>? I think that would work for me.
> I meant in the metadata. I don't see why you would want to link to a bug about a test inside a test, really.

This *is* metadata :) Storing the information in the test means that when you fix the bug, you just remove the link and any system processing the test can immediately see that it is flagged as bug-free. If you separate the metadata from what it applies to, it is likely to go out of sync and we'll end up with tests that have been fixed but not flagged as such in the system.

In my ideal world all the useful information for a given test would be contained in that test. That way we can have multiple orthogonal systems spider the tests with the only centralised information being the list of known tests (or directories containing tests). This would provide us with orthogonality and avoid having to integrate too much disparate information into a single framework.

I'm saying this because I'm concerned that the current framework has to address very varied needs  just looking at how CSS, WebApps, ARIA, I18N use it I see it needing a lot of flexibility, and that's hard to provide within a single code base. I don't at all want to upset the apple cart, but I'm increasingly getting the impression that it would be easier to build testing frameworks atop multiple smaller services with public APIs.

Robin Berjon - http://berjon.com/ - @robinberjon
Received on Monday, 4 June 2012 16:58:55 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:08 UTC