Re: Testable assertion tagging for W3C specifications

Scott,

Being an editor of the Specification Guidelines document in the QA 
activity, where I advocate this, I'm happy to see that more people are 
thinking about these things. In the document (see 
http://www.w3.org/QA/WG/2002/04/qaframe-spec-0429.html for the latest 
draft), you'll find similar ideas that definitely need to be worked on. 
I look forward to your input and will provide comments on your work.

best,

/Dimitris



On Monday, May 6, 2002, at 07:29 , scott_boag@us.ibm.com wrote:

> A colleague and I have been talking about how to do more precise 
> testing of
> W3C specifications, and how the specification markup might help.
>
> GOALS:
>    Allow an external document (test case, erratum, email, etc.) to point
>    directly at a "testable" normative sentence in a Recommendation.
>    Encourage document editors to view some of the sentences as "test
>    assertions" and to write them in a style that conveys precisely what
>    they declare.
>    Explore possibilities for machine processing of testable sentences in
>    the future.
>    Link error assertions to error catalogues (see the work that Mike 
> Kay is
>    doing with the XSLT document: (
>    http://www.w3.org/TR/xslt20/#error-summary)).
>    Provide a tagging scheme for testing of grammatical statements, such 
> as
>    the ad-hoc one employed in the XPath/XQuery specifications.
>    Possibly provide markup also for discretionary behavior.
>
> So our proposal is to add a tagging structure to
> http://www.w3.org/XML/1998/06/xmlspec-report.htm,
> http://www.w3.org/XML/1998/06/xmlspec.dtd, that allows editors to 
> achieve
> the above goals.
>
> Test cases will nearly always have to cite more than one testable 
> sentence
> and/or production, unless the Rec is issued with test assertions in a
> separate appendix. We should experiment with enhanced tagging to see 
> how it
> influences sentence structure. Some complex sentences with multiple "or"
> parts crossing each other may get restructured just to make citing them
> more precise. Consider this sentence from part 16.4 of XSLT 1.0: "Thus, 
> it
> is an error to disable output escaping for an xsl:value-of or xsl:text
> element that is used to generate the string-value of a comment, 
> processing
> instruction or attribute node; it is also an error to convert a result 
> tree
> fragment to a number or a string if the result tree fragment contains a
> text node for which escaping was disabled." That one sentence has 8-10
> testable assertions.
>
> So far, we are not proposing concrete details.  First we wanted to see 
> what
> people thought of the idea, if anyone has experimented with something 
> like
> this so far, and whether or not this would be worth a concrete proposal.
>
> -scott
>
>
>
>

Received on Monday, 6 May 2002 17:00:48 UTC