[widgets] Test Suite Creation

The Widgets P&C document will be "officially" published tomorrow (23rd
of July). The way of progressing the Widgets P&C document to PR is to
have two implementations that pass every test in the test suite, it is
now really important to have a test suite that fully tests every
relevant aspect of the spec.

In preparation, I've created a Overview.html skeleton file to begin
describing the structure of the test suite and what is tested:

http://dev.w3.org/2006/waf/widgets/tests/

As editor of the spec, I would like to work with the MWTS WG in
creating an suitable test suite for implementers.

My recommendation for going forward is:

(I know some of the work below has already been done, but, there are
aspects that I'm uncertain about.)

1. Use Dom's tool for extracting testable assertions in the spec
(http://tinyurl.com/ls5zsl) and evaluate that assertions are written
correctly: this means reducing any variability in the assertions in
the manner described in http://www.w3.org/TR/spec-variability/.

2. Fix any bogus assertions - break up assertions into classes of
products (and weed out untestable assertions that apply only to
authors). This will be editorial changes to P&C.

(Dom: could your XSLT be modified to group together assertions by
class of product? I could change the markup to identify products
really easily: e.g., "user agent" becomes <span class="product
ua">user agent</span> or whatever would work best for you)

3. Settle on a Stable Editors' Draft that will become the source for
generating the tests.

4. Create a table of stable testable assertions (broken up by class of
product).

5. Design the template, naming convention, and metadata tests.

6. Create the tests for each assertion (How do we verify the test
actually covers the each assertion?).

WDYT?

Kind regards,
Marcos

-- 
Marcos Caceres
http://datadriven.com.au

Received on Wednesday, 22 July 2009 15:56:24 UTC