Re: Testable assertion tagging for W3C specifications


Being an editor of the Specification Guidelines document in the QA 
activity, where I advocate this, I'm happy to see that more people are 
thinking about these things. In the document (see for the latest 
draft), you'll find similar ideas that definitely need to be worked on. 
I look forward to your input and will provide comments on your work.



On Monday, May 6, 2002, at 07:29 , wrote:

> A colleague and I have been talking about how to do more precise 
> testing of
> W3C specifications, and how the specification markup might help.
>    Allow an external document (test case, erratum, email, etc.) to point
>    directly at a "testable" normative sentence in a Recommendation.
>    Encourage document editors to view some of the sentences as "test
>    assertions" and to write them in a style that conveys precisely what
>    they declare.
>    Explore possibilities for machine processing of testable sentences in
>    the future.
>    Link error assertions to error catalogues (see the work that Mike 
> Kay is
>    doing with the XSLT document: (
>    Provide a tagging scheme for testing of grammatical statements, such 
> as
>    the ad-hoc one employed in the XPath/XQuery specifications.
>    Possibly provide markup also for discretionary behavior.
> So our proposal is to add a tagging structure to
>, that allows editors to 
> achieve
> the above goals.
> Test cases will nearly always have to cite more than one testable 
> sentence
> and/or production, unless the Rec is issued with test assertions in a
> separate appendix. We should experiment with enhanced tagging to see 
> how it
> influences sentence structure. Some complex sentences with multiple "or"
> parts crossing each other may get restructured just to make citing them
> more precise. Consider this sentence from part 16.4 of XSLT 1.0: "Thus, 
> it
> is an error to disable output escaping for an xsl:value-of or xsl:text
> element that is used to generate the string-value of a comment, 
> processing
> instruction or attribute node; it is also an error to convert a result 
> tree
> fragment to a number or a string if the result tree fragment contains a
> text node for which escaping was disabled." That one sentence has 8-10
> testable assertions.
> So far, we are not proposing concrete details.  First we wanted to see 
> what
> people thought of the idea, if anyone has experimented with something 
> like
> this so far, and whether or not this would be worth a concrete proposal.
> -scott

Received on Monday, 6 May 2002 17:00:48 UTC