Re: Testable assertion tagging for W3C specifications

Hi,

yes, the WAI Evaluation and Repair  tools group has experimented to the
extent of developing an RDF framework and having tools that use it.

The framework is called EARL, and there is some introductory information at
http://www.w3.org/2001/03/earl (but that is a bit out of date now) and more
information linked from the ER group page http://www.w3.org/WAI/ER

It turns out that we need to have a couple of things - one is a way of
describing what the test is (we use a URI to refer to it - in the initial use
case of WAI specifications that was OK because there were specific
checkpoints with URIs) and another is enough logic to make compound
statements such as "because X passes requirements Z and Q it passes
requirement F" (where there is some description that says "passing (x or Y or
Z) and [(P or Q) or (A and B)] implies passing requirement F"). There are a
couple of pieces of software dealing with this now.

cheers

Charles McCN

On Mon, 6 May 2002 scott_boag@us.ibm.com wrote:

  A colleague and I have been talking about how to do more precise testing of
  W3C specifications, and how the specification markup might help.

  GOALS:
     Allow an external document (test case, erratum, email, etc.) to point
     directly at a "testable" normative sentence in a Recommendation.
     Encourage document editors to view some of the sentences as "test
     assertions" and to write them in a style that conveys precisely what
     they declare.
     Explore possibilities for machine processing of testable sentences in
     the future.
     Link error assertions to error catalogues (see the work that Mike Kay is
     doing with the XSLT document: (
     http://www.w3.org/TR/xslt20/#error-summary)).
     Provide a tagging scheme for testing of grammatical statements, such as
     the ad-hoc one employed in the XPath/XQuery specifications.
     Possibly provide markup also for discretionary behavior.

  So our proposal is to add a tagging structure to
  http://www.w3.org/XML/1998/06/xmlspec-report.htm,
  http://www.w3.org/XML/1998/06/xmlspec.dtd, that allows editors to achieve
  the above goals.

  Test cases will nearly always have to cite more than one testable sentence
  and/or production, unless the Rec is issued with test assertions in a
  separate appendix. We should experiment with enhanced tagging to see how it
  influences sentence structure. Some complex sentences with multiple "or"
  parts crossing each other may get restructured just to make citing them
  more precise. Consider this sentence from part 16.4 of XSLT 1.0: "Thus, it
  is an error to disable output escaping for an xsl:value-of or xsl:text
  element that is used to generate the string-value of a comment, processing
  instruction or attribute node; it is also an error to convert a result tree
  fragment to a number or a string if the result tree fragment contains a
  text node for which escaping was disabled." That one sentence has 8-10
  testable assertions.

  So far, we are not proposing concrete details.  First we wanted to see what
  people thought of the idea, if anyone has experimented with something like
  this so far, and whether or not this would be worth a concrete proposal.

  -scott





-- 
Charles McCathieNevile    http://www.w3.org/People/Charles  phone: +61 409 134 136
W3C Web Accessibility Initiative     http://www.w3.org/WAI  fax: +33 4 92 38 78 22
Location: 21 Mitchell street FOOTSCRAY Vic 3011, Australia
(or W3C INRIA, Route des Lucioles, BP 93, 06902 Sophia Antipolis Cedex, France)

Received on Monday, 6 May 2002 14:48:18 UTC