FW: Updated CR test overview page

If you have not had chance or saw the work that Robin has done below please take note and provide feedback.

-Thx
________________________________________
From: Robin Berjon <robin@w3.org>
Sent: Tuesday, December 17, 2013 9:12 AM
To: HTML WG (public-html@w3.org)
Subject: Updated CR test overview page

Hi all,

I've produced an updated version of the testing page. I wouldn't call it
done — there are still some things that I want to add to it — but it's
changed enough that feedback can be usefully made.

You can find it at:

     http://www.w3.org/html/wg/tests-cr-exit/

Some notes:

   • Hopefully the labels make it clearer; suggestions for improvements
welcome.
   • One thing I'm adding is filtering, to make it easy to see just a
subset.
   • The "priority" items come from the list that Erika and Denis made.
   • The number of tests are *very* underestimated, this for several
reasons:
     - Many tests test several sections but are only categorised in one
(much of chapter 2 is covered in that way)
     - In a number of cases, tests that could apply to a subsection have
been categorised as belonging to the containing chapter (so for instance
the subsections of the parsing chapter tend to have 0 tests but are in
fact covered by the extensive testing done at the chapter's root)
     - A number of sections are not marked as non-normative but have no
effective normative content.
     - A large chunk of tests are still being reviewed or in the
backlog, and so don't appear at all. We're catching up on those, though.
     - The tool that gathers the number of tests automatically is still
very flimsy. It registered numerous crashes in the latest run I had it
perform. This information will generally get better, but for the moment
it is erring heavily on the side of undercounting.

So all in all there's still work, but it's in better shape (and on saner
foundations than the previous stab).

One of the next steps is to go through the sections that flagged as
having to be tested to perform a direct estimation of how well-tested it
is (i.e. remove the problems with automatic coverage estimation from the
equation), and when good flag those sections as "ok". Then when
everything is ok we ship :)

--
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Wednesday, 15 January 2014 19:50:42 UTC