Re: [widgets] Test Suite Creation

Hi,
A brief update on the status of this...

On Wed, Jul 22, 2009 at 5:55 PM, Marcos Caceres<marcosc@opera.com> wrote:
> The Widgets P&C document will be "officially" published tomorrow (23rd
> of July). The way of progressing the Widgets P&C document to PR is to
> have two implementations that pass every test in the test suite, it is
> now really important to have a test suite that fully tests every
> relevant aspect of the spec.
>
> In preparation, I've created a Overview.html skeleton file to begin
> describing the structure of the test suite and what is tested:
>
> http://dev.w3.org/2006/waf/widgets/tests/
>
> As editor of the spec, I would like to work with the MWTS WG in
> creating an suitable test suite for implementers.
>
> My recommendation for going forward is:
>
> (I know some of the work below has already been done, but, there are
> aspects that I'm uncertain about.)
>
> 1. Use Dom's tool for extracting testable assertions in the spec
> (http://tinyurl.com/ls5zsl) and evaluate that assertions are written
> correctly: this means reducing any variability in the assertions in
> the manner described in http://www.w3.org/TR/spec-variability/.
>
> 2. Fix any bogus assertions - break up assertions into classes of
> products (and weed out untestable assertions that apply only to
> authors). This will be editorial changes to P&C.
>
> (Dom: could your XSLT be modified to group together assertions by
> class of product? I could change the markup to identify products
> really easily: e.g., "user agent" becomes <span class="product
> ua">user agent</span> or whatever would work best for you)


I've created the first draft of the test suite edition, it is
available here (it's ugly on purpose, I will remove the ugly
stylesheet soon):

http://dev.w3.org/2006/waf/widgets/Overview_TSE.html

Includes stable IDs on <p>'s, and HTML-class-based identification of
products to which assertions apply:

either: "product-cc" or "product-ua"

> 3. Settle on a Stable Editors' Draft that will become the source for
> generating the tests.

In the last teleconf, we agreed we would fix things iteratively.

> 4. Create a table of stable testable assertions (broken up by class of
> product).

This can probably begin now (i.e.,
http://dev.w3.org/2006/waf/widgets/tests/plan.html can now be
updated).

> 5. Design the template, naming convention, and metadata tests.

Will work with Kai on this.

> 6. Create the tests for each assertion (How do we verify the test
> actually covers the each assertion?).

Will work with Kai on this... Opera _may_ submit some tests.

-- 
Marcos Caceres
http://datadriven.com.au

Received on Friday, 31 July 2009 16:02:36 UTC