Re: Testable assertion tagging for W3C specifications

I'll settle in on the central theme of Alex's latest reply.

>I think that if I can address any arbitrary piece of a Rec, I can do
>everything I need.

Well, sure, if the authors were careful enough to say everything
they should have said.

Alex's idea seems to be that we devise a system of pointing down to
individual characters, treating the whole Rec as a byte stream.
If you need more than one range, cite more than one.

I say if that's enough for him, then he's better off to cut/paste
the actual text into the test case or other place where needed.
Two benefits: immunity to byte shifts in the source, and no need to
develop the necessary XPointer improvements. Naturally, the worry
is that your extracted text may diverge from what's normative as
errata are issued.

We don't want to address arbitrary text ranges. We want to address
meaningful (and normative) syntactic units. QAWG, through the Spec
Guidelines, would encourage authors to add tags that make these
syntactic units more evident. The person citing should not be
given a tool for selecting arbitrary text ranges and left to their
own devices to produce "good" citations, but should rather be
given tags that are part of the expressive power used by the spec
authors to convey their intent.

>For example, when a user clicks on a test case description, the
>description may include some readable narrative _and_ precise
>quote(s) from the Recommendation that, together, give user a good
>idea on what is being tested (among other details).

That's our current practice, but the quotes are obtained by cut
and paste from the normative documents. I can drag my cursor over
exactly the range of text I want. So can anyone else. But if
people from different organizations are contributing test cases
into a common effort, this approach maximizes the ability for
their different interpretations to clash. Yet the Rec authors know
(or should know) exactly which sentences (or productions, etc.)
are supposed to be the governing ones for any situation.

The hard part, which I will state as a requirement because Alex
asked for requirements, is to express that a test case is only
testing a particular combination of circumstances. If I have one
test case of <xsl:number level="single" count="foo"/>, I have a
combination of both stated and defaulted choices, and I do NOT
have adequate coverage of level="single" due to the other
combinations in which it could be a part. I think that area of
the spec has thin verbiage, so I can't point at all the sentences
I need, because some aren't even there. If the Rec authors had
tagging requirements that drove them to consider coverage of the
combinations, they would better serve the needs of QA and
developers alike.
.................David Marston

Received on Wednesday, 29 May 2002 23:50:28 UTC