LC-67 leftover -- MUST use MUST?

This a leftover issue about test assertions and conformance 
requirements.  Let's have some mail discussion so that the SpecGL editors 
can close it at next telecon.

The first part of LC-67 [1] asks, "must all [conformance requirements] use 
RFC2119 keywords?"

[Note.  In that paraphrasing, I substituted "conformance requirements" for 
the originator's "testable statements and or test assertions", in line with 
our clarification of terminology at the Crete face-to-face.  By our 
definition of test assertions, they inherently do not use the RFC2119 
keywords.]

This was originally addressed, including a proposed resolution, in email 
[2].  It was further discussed a bit at the Crete f2f without resolution [3].

Alternatives:
-----

At the face-to-face, these alternatives seemed most popular:

Alt.1:  yes, per the current wording of the (priority 1) SpecGL CP13.1 [4] 
and its rationale, all specifications must use the RFC2119 keywords [in the 
statement of conformance requirements].

Alt.2:  no, the RFC2119 keywords are the recommended and preferred way, but 
other language is permitted (e.g., marked-up imperative voice statements), 
as long as the spec unambiguously defines what are its conformance 
requirements and how are they identified in the text.

Alt.3:  as Alt.3, but the spec must particularly define the correspondence 
between its normative language scheme and the RFC2119 keywords.

Discussion
-----

Among the points made in f2f discussion:

** Alt.2 or Alt.3 strongly encourage uniformity rather than forcing it, and 
that should suffice.

** Contra.  Uniformity should be mandated for all new specifications (and 
don't be too concerned with SpecGL conformance for existing specifications.)

** Observation.  UAAG10 is almost AAA-conformant, but will fail CP13.1 [4] 
as written and so would not be A-conformant (UAAG10 fits Alt.3 pretty 
well).  Some thought that it would hurt SpecGL credibility, if it fails a 
spec that is generally considered pretty good.

** Note.  QAWG asked UAWG about its rationale for not purely following 
RFC2119, and UAWG replied -- see [5].

** Difficulty with CP13.1 as written:  what about specs in formal language 
like DOM, or specs that use embedded test assertions for normative text, 
instead of more general conformance requirements?  (Note.  Are there any 
good examples of the latter?)

** Pro Alt.1:  per the rationale at [4], this forces spec writers to 
unambiguously indicate what are the conformance requirements, and makes it 
very easy for implementers, test builders, etc to find the conformance 
requirements.

Discussion and/or proposals?

Regards,
-Lofton.

[1] http://www.w3.org/QA/WG/lc-issues#x67
[2] http://lists.w3.org/Archives/Public/www-qa/2003Jun/0013.html
[3] http://lists.w3.org/Archives/Public/www-qa-wg/2003Jun/0041.html
[4] http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/#Ck-use-keywords
[5] http://lists.w3.org/Archives/Public/www-qa-wg/2003Jun/0039.html

Received on Wednesday, 25 June 2003 11:54:23 UTC