Spec Guideline: subjectivity (re: issue #91)

Issue #91 on the issues list at
http://www.w3.org/QA/WG/qawg-issues-html
questions Checkpoint 8.4 of the Spec Guidelines (SpecGL). That
checkpoint says:

Checkpoint 8.4. Include a statement regarding consistent handling of a
discretionary item within an implementation. [Priority 2]
The effect of each individual discretionary item should be consistent
within a single implementation. For example, a browser's rendering of a
XSL-FO (XSL Formatting Object) should be the same for every invocation
regardless of the document instance.

I think the checkpoint needs rewriting. The thrust of it is that any
individual discretionary item is one that the implementer must decide
once for the entire implementation. If the spec allows the choice of
behaviors A or B (but no others), then an implementation must not make
the product sometimes do A and sometimes B, depending on other factors.
The spec, however, can recognize the other factors and define more than
one discretionary item if it is convinced that such is a split provides
more benefits than costs.

The guideline is worthwhile, but the explanatory matter should point
out why: so that a test case can be tied to a particular choice. The
creators of the test suite can work from the spec to produce separate
cases for A and B. An implementer can choose either A or B, and if they
use the test suite, they pick one set of tests (the A set or the B set)
to check their work. A vendor-neutral testing lab that is testing more
than one implementation can apply different sets to the different
implementations, according to the Implementation Conformance
Statements (GL 12, Issue #96) from each vendor. This is yet another
example of how the SpecGL requires spec writers to write specs that
are testable, not just because the content is testable, but also because
the spec sets up conditions that allow objective application of the
tests that should be coming from the WG (or its designee) as another
deliverable.
.................David Marston

Received on Monday, 7 October 2002 16:56:59 UTC