Re: Test Case Approval Request

On 04/06/2010 04:36 PM, Kris Krueger wrote:
> This is only one of the 19 tests that have been submitted to HG.  I picked this one because it's correct per the specification and should not be controversial.
>
> For example some of the other HTML DOM cases are 'obsolete' (e.g. anchor02), so they should never get approved.
> Other cases in the submitted folder are incorrect per the HTML5 spec and will need updates before they can get submitted.
>    

I understand the desire to test the processes here, but I would be 
significantly more inclined to spend time reviewing a small testsuite 
than an individual test. With a small testsuite it is possible to see 
how systematically the tests have been written , whether they completely 
cover an individual section of the spec, and so on. With a single 
isolated test one can't really say any of those things only "this is 
right per spec" or "this is wrong per spec". It seems sufficiently 
harder to get from the latter to a coherent view of whether we have good 
test coverage for the spec as a whole that we should avoid accepting 
individual (standalone) tests but instead ask that individual tests are 
written to fit in with some preexisting suite that covers the part of 
the spec in question. For bootstrapping purposes that means we must 
start by trying to write and review testsuites rather than individual tests.

Received on Tuesday, 6 April 2010 15:06:49 UTC