Updated CR test overview page

Hi all,

I've produced an updated version of the testing page. I wouldn't call it 
done — there are still some things that I want to add to it — but it's 
changed enough that feedback can be usefully made.

You can find it at:

     http://www.w3.org/html/wg/tests-cr-exit/

Some notes:

   • Hopefully the labels make it clearer; suggestions for improvements 
welcome.
   • One thing I'm adding is filtering, to make it easy to see just a 
subset.
   • The "priority" items come from the list that Erika and Denis made.
   • The number of tests are *very* underestimated, this for several 
reasons:
     - Many tests test several sections but are only categorised in one 
(much of chapter 2 is covered in that way)
     - In a number of cases, tests that could apply to a subsection have 
been categorised as belonging to the containing chapter (so for instance 
the subsections of the parsing chapter tend to have 0 tests but are in 
fact covered by the extensive testing done at the chapter's root)
     - A number of sections are not marked as non-normative but have no 
effective normative content.
     - A large chunk of tests are still being reviewed or in the 
backlog, and so don't appear at all. We're catching up on those, though.
     - The tool that gathers the number of tests automatically is still 
very flimsy. It registered numerous crashes in the latest run I had it 
perform. This information will generally get better, but for the moment 
it is erring heavily on the side of undercounting.

So all in all there's still work, but it's in better shape (and on saner 
foundations than the previous stab).

One of the next steps is to go through the sections that flagged as 
having to be tested to perform a direct estimation of how well-tested it 
is (i.e. remove the problems with automatic coverage estimation from the 
equation), and when good flag those sections as "ok". Then when 
everything is ok we ship :)

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Tuesday, 17 December 2013 17:12:35 UTC