Re: Review of Test Guidelines

Note: I don't subscribe to www-qa, but posted my review comments here as
requested.

Please explicitly include me on the to or cc line for discussions of my
comments.


>
Jeremy Carroll:
>>This suggests a priority 1 guideline that there should be a
>>recommendation track document for the test work in a WG.
David Marston:
>Disagree. The test suite applies to a Rec-track document, but if it
>were a separate document, you would have synchronization problems.
>The actual test case collection needs to be filterable, and will be
>able to be filtered for various versions of the Rec, as well as the
>other Dimensions of Variability.

I have accepted Karl's similar disagreement here.
I think the (rec) versioning issues can be addressed simply by creating zip
files. Other dimensions of variability probably need addressing in test
metadata.

David Marston:
>Test case writers can be assigned to try to find
>holes in the current Working Draft, which may require creativity.

A practical problem is to do with personalities; e.g. Ian Horrocks in WebOnt
is one of the most practically and theoretically skilled members of the
group. He has sketched a test case that demonstrates a work-around for an
issue we are discussing:
http://lists.w3.org/Archives/Public/www-webont-wg/2003Jul/0157.html
[[
E.g., for Jim's reptiles, we
could add:

SubClassOf(Reptile  restriction(reptile-name cardinality(1)))

and for each reptile something like:

SubClassOf(Crocodile restriction(reptile-name value("Crocodile")))


Regards, Ian
]]

To make this into the test case we need to convert into RDF and then into
RDF/XML and then expand the "for each" above, say three times over.

This is basically dull, which is why Ian did not do it.

Ian has done the fun bit, and someone else (e.g. me) will end up doing the
drudgery. That sounds like a complaint, which it's not meant to be. I doubt
I could have found this construct; and Ian is not as skilled as me at the
RDF/XML part which is still to do.

In terms of David's point, while there is fun to be had in testing, there is
a lot of drudge work too. Perhaps we need to set our sights low enough to
make the drudgery bearable, taking into account where we think the tests are
coming from - which may well vary from WG to WG and from activity to
activity.

Jeremy:
>>...Requiring documentation is the instrument of mediocrity.
David:
>Ouch.

I was commenting on a specific requirement for documenting a plan.
It seemed much more pertinent to me to require the plan to succeed than to
be documented!

Documentation can be beaureaucracy for beaureaucracy's sake.

Jeremy

Received on Monday, 21 July 2003 09:51:34 UTC