Re: classify every section in the spec

Kris Krueger wrote:
> Notes
> 
> The group discussed how to leverage robins coverage report such that
> we start to enumerate sections of the spec that we should prioritize
> for testing.
> We agreed that that the task force should start to classify every
> section in the spec, such that each section would be labeled.
> 
> For example...
>     A) no requirements
>     B) has conformance requirements, no tests, known interop issues (for example the-input-byte-stream)
>     C) has conformance requirements, has tests (canvas)
>     D) has conf reqs, no tests, no known interop issues
> 
> http://w3c-test.org/html-testsuite/master/tools/coverage/

To expand on something I was saying during the meeting...

While I agree that it's useful to know which of these classes each
section is in, I'm doubtful that the best way to do this is to have
humans assigning one of these 4 labels to each section (which may or may
not be what Kris meant).

The classes above are synthesized from 3 more basic conditions:
  (1) whether the section has conformance requirements;
  (2) whether there are tests for that section; and
  (3) whether there are known interop issues pertaining to that section.

Note also that these conditions can change when (respectively):
  (a) the spec is edited;
  (b) tests are submitted or edited; or
  (c) a new version of a browser is released.

Robin's coverage report already tells us (1) and (2), and (I gather) can
be regenerated at will to reflect changes due to (a) and (b).

Thus, rather than classifying sections into A/B/C/D, you could get the
same information with less work (both upfront and ongoing) by just
classifying them wrt (3), presence/level of known interop issues.

(I'm assuming this has to be done by humans, i.e., you can't easily write
a script to deduce the level of known interop issues for a section. Can
you? I wondered whether we could import data from caniuse.com, but Kris
said no.)

-Michael

Received on Thursday, 28 February 2013 02:13:02 UTC