Re: Testable assertion tagging for W3C specifications

Please use this version,

[2] http://www.w3.org/QA/WG/2002/framework-20020506/qaframe-spec.html

instead of Dimitris' reference.  Mostly editorial improvements on the path 
to publication, but significant ones there.  Note that these are not stable 
references, as they are editor's working copies!  (Okay, having pointed to 
it, we can leave it as is for a while.)

-Lofton.

At 11:00 PM 5/6/2002 +0200, Dimitris Dimitriadis wrote:
>Scott,
>
>Being an editor of the Specification Guidelines document in the QA 
>activity, where I advocate this, I'm happy to see that more people are 
>thinking about these things. In the document (see 
>http://www.w3.org/QA/WG/2002/04/qaframe-spec-0429.html for the latest 
>draft), you'll find similar ideas that definitely need to be worked on. I 
>look forward to your input and will provide comments on your work.
>
>best,
>
>/Dimitris
>
>
>
>On Monday, May 6, 2002, at 07:29 , scott_boag@us.ibm.com wrote:
>
>>A colleague and I have been talking about how to do more precise testing of
>>W3C specifications, and how the specification markup might help.
>>
>>GOALS:
>>    Allow an external document (test case, erratum, email, etc.) to point
>>    directly at a "testable" normative sentence in a Recommendation.
>>    Encourage document editors to view some of the sentences as "test
>>    assertions" and to write them in a style that conveys precisely what
>>    they declare.
>>    Explore possibilities for machine processing of testable sentences in
>>    the future.
>>    Link error assertions to error catalogues (see the work that Mike Kay is
>>    doing with the XSLT document: (
>>    http://www.w3.org/TR/xslt20/#error-summary)).
>>    Provide a tagging scheme for testing of grammatical statements, such as
>>    the ad-hoc one employed in the XPath/XQuery specifications.
>>    Possibly provide markup also for discretionary behavior.
>>
>>So our proposal is to add a tagging structure to
>>http://www.w3.org/XML/1998/06/xmlspec-report.htm,
>>http://www.w3.org/XML/1998/06/xmlspec.dtd, that allows editors to achieve
>>the above goals.
>>
>>Test cases will nearly always have to cite more than one testable sentence
>>and/or production, unless the Rec is issued with test assertions in a
>>separate appendix. We should experiment with enhanced tagging to see how it
>>influences sentence structure. Some complex sentences with multiple "or"
>>parts crossing each other may get restructured just to make citing them
>>more precise. Consider this sentence from part 16.4 of XSLT 1.0: "Thus, it
>>is an error to disable output escaping for an xsl:value-of or xsl:text
>>element that is used to generate the string-value of a comment, processing
>>instruction or attribute node; it is also an error to convert a result tree
>>fragment to a number or a string if the result tree fragment contains a
>>text node for which escaping was disabled." That one sentence has 8-10
>>testable assertions.
>>
>>So far, we are not proposing concrete details.  First we wanted to see what
>>people thought of the idea, if anyone has experimented with something like
>>this so far, and whether or not this would be worth a concrete proposal.
>>
>>-scott
>>
>>
>>
>