W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2011

Example of "automatic" extraction of testable assertions

From: Francois Daoust <fd@w3.org>
Date: Thu, 03 Mar 2011 15:48:36 +0100
Message-ID: <4D6FAA44.70307@w3.org>
To: "Michael(tm) Smith" <mike@w3.org>
CC: public-test-infra@w3.org
Hi Mike,

Mostly FYI and FWIW, as we say.

I mentioned I had played a bit with XSLT for extract test assertions from the Guidelines for Web Content Transformation Proxies. The code is available at:
  http://dev.w3.org/cvsweb/2010/ct-guidelines/test-suite/tools/

The part that may be of interest is the XSLT that generates IDs for each of the testable assertion:
  http://dev.w3.org/cvsweb/2010/ct-guidelines/test-suite/tools/generate-ta-ids.xsl

With a few hardcoded exceptions, it simply extracts sentences around RFC2119 keywords in the spec and generate an ID for each of them in the form of an hash of the normative sentences. The spec, completed with IDs, can then be used to extract test assertions and create a companion test suite (different XSLT) spec, or an implementation conformance statement:
  http://dev.w3.org/2010/ct-guidelines/test-suite/Overview.html
  (no test cases linked to that spec since work on the spec got stopped at this point...)
  http://www.w3.org/TR/ct-guidelines/ics.html

Obviously, when a sentence changes, the ID changes, so that's not very resistant. The HTML5 spec is also much more complex and testable assertions cannot be identified using occurrences of RFC2119 keywords. Regular expressions will be needed, as discussed. I thought I'd share it anyway, as example.

[On a side note, one thing we haven't mention yet is that there are different classes of products in the HTML5 spec, and we're very likely going to need to know which testable assertion belongs to which (set of) class(es) of product]

Francois.
Received on Thursday, 3 March 2011 14:49:09 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 3 March 2011 14:49:10 GMT