Conformance requirement markup and test metadata (take 2)

After the feedback on this list and discussing the idea with Elika, Peter
and Vincent, I have a new plan for adding conformance requirements markup to
the css3-regions spec and referring to that spec markup in test metadata.

Test files currently have a reference to the section of the spec they test.
We'd like to be a bit more specific about what each test covers. I will be
adding metadata to Regions tests that includes any relevant id in the spec.
This might just be the id of the section the test already refers to. Or it
could be more specific, perhaps referring to one or more <dfn> ids.

For the Regions spec we will likely choose to add more ids to the spec (at
the paragraph or sentence level) to further categorize the tests, but this
scheme could be used without adding any additional markup to specs for
existing testable ids.

The goal is to be able to scan the spec for testable ids and check whether
each testable id has at least one test. So there will need to be a set of
rules for determining which ids in a spec are the testable ones, and which
can be ignored in the scan. Whether an id is testable can be determined by
scanning the spec and ignoring any id in elements with these class
attributes: example, note, no-toc or informative. The "informative" class
attribute is new, and can be added around non-normative sections of the
spec, or any id that's found not to correspond to a conformance requirement.

I've added this plan to the css3-regions wiki
http://wiki.csswg.org/spec/css3-regions and plan on using this scheme for
css3-regions tests. I assume we may find additional heuristics for finding
testable ids as we go. Any and all comments and suggestions are welcome.

Thanks,

Alan

Received on Thursday, 1 September 2011 23:49:33 UTC