Re: Review of Test Guidelines

Responding to Jeremy Carroll's message of 1 July 2003:

>This is a review of the Test Guidelines WD dated 16 May 2003

>It is important to identify what motivates the best people to
>participate in the W3C and what demotivates them. It is at least
>plausible that peer group acclaim is important, and hence it is
>important that WG members who contribute to the test work are
>adequately acknowledged.

Agreed. (...he said, self-servingly)

>This suggests a priority 1 guideline that there should be a
>recommendation track document for the test work in a WG.

Disagree. The test suite applies to a Rec-track document, but if it
were a separate document, you would have synchronization problems.
The actual test case collection needs to be filterable, and will be
able to be filtered for various versions of the Rec, as well as the
other Dimensions of Variability.

>A good choice of editor who takes pride in their work, will be one of
>the best ways to ensure a deep quality to the test work.

True whether you're referring to an editor or to a test-collection
supervisor who takes pride in the suite's coverage.

>It may be appropriate that each test is individually attributed.

Agreed.

>...As it is, the waterfall model you suggest looks like makework for
>mediocre members of the WG

Only as you've read more waterfall methodology into the document than
is actually there. Test case writers can be assigned to try to find
holes in the current Working Draft, which may require creativity.

>...Requiring documentation is the instrument of mediocrity.

Ouch. I thought it was the instrument of interoperability and the
way that non-WG vendors would have some hope of producing their
implementations.

BTW, when a commitment level requires that a particular resource be
delivered as a requirement to go to Rec, that is not enforcing the
waterfall methodology. **Everything** should be finished. You can
anticipate that errata will be needed (due to human nature), but the
resources delivered for the Rec should have no known errors. I think
that QAWG will clarify this in later drafts.
.................David Marston

Received on Friday, 18 July 2003 00:05:30 UTC