Conformance requirement markup and test metadata

(Cross-posted to www-style. Please reply to public-css-testsuite)

Vincent, Peter and I have been talking about adding more information to the
Regions spec for testing. Right now, test cases have metadata that points
back to the relevant section of the spec. We're planning on adding markup to
the spec for each individual conformance requirement and having the tests
also refer to the specific requirement they test.

As an example, there is a sentence in the Regions spec that states:

  The values "none", "inherit", "default" and "initial" are invalid flow
names.

This will be marked up as a conformance requirement with class="conform" and
a unique id:

  <span class="conform" id="invalid-flow-idents">The values "none",
"inherit", "default" and "initial" are invalid flow names.</span>

Then one or more test cases that address this conformance requirement will
contain additional metadata with that id:

  <meta name="for" content="invalid-flow-idents"/>

(the "for" name and requirement ids come from
http://www.w3.org/TR/test-methodology/, and class="conform" comes from usage
in the WOFF spec)

This will give us a finer-grained notion of what each test is for, and allow
us to ensure that each conformance requirement in the spec has at least one
test associated with it (via a tool like Shepherd instead of manual
inspection).

Please let us know what you think of this proposal, if there are
modifications you'd recommend, and whether you would be interested in
following this practice in your own spec(s).

Thanks,

Alan

Received on Friday, 19 August 2011 23:05:26 UTC