- From: James Graham <jgraham@opera.com>
- Date: Fri, 17 Aug 2012 22:51:55 +0200 (CEST)
- To: "L. David Baron" <dbaron@dbaron.org>
- cc: Boris Zbarsky <bzbarsky@mit.edu>, public-html@w3.org
On Fri, 17 Aug 2012, L. David Baron wrote: > On Thursday 2012-08-16 10:32 +0200, James Graham wrote: >> On 08/16/2012 09:21 AM, Boris Zbarsky wrote: >>> Well, a good start would be having a simple, advertized way to >>> contribute tests. I don't even know how I'd go about contributing a >>> test for the HTML5 test suite, and my experience with contributing tests >>> for other W3C working groups has been less than pleasant. :( >>> >>> Of course contributing test suites to browsers that are not Mozilla is >>> not that easy for me either. But for Mozilla, it takes me very little >>> time to add a test to the Mozilla test suite. If we can get >>> contributing tests for the official test suite down to that level, I bet >>> developers will be more likely to contribute. >> >> Well the actual "contribute" part is basically "commit to a hg >> repo". The full instructions are at [1]. > > Are those instructions sufficient to get a test into the "Approved > Tests" list? [2] Or is the "Approved Tests" subset not a relevant > subset? My opinion is that the approval system we have has not worked. Where we have imported tests at Opera we typically run them just from the submitted directory because that contains far more tests, and by actually running them we are more likely to find problems than by simply inspecting them (or waiting for others to do so). In the long term I think we should remove the submitted/ and approved/ directories altogether; the filesystem structure is the wrong place to store review metadata. Of course I still think that code review is valuable, but I would mcuh rather integrate with the VCS and say, for example, that specific commits, or (file, commit) tuples have been reviewed. Also, compared to e.g. the CSS working group, the number of formal requirements for tests has been kept intentionally low; although it is possible to add similar types of metadata it is not required. Thinking about it, an unintentional benefit of dropping the testsuite from the Process requirements is that it will become easier to submit tests because there will be less pressure to add requirements on test authors desinged to make getting implementation reports easy.
Received on Friday, 17 August 2012 20:52:37 UTC