Mark's comments on Checkpoint 1.1 Table

>1) We've asked the WG to commit to a test suite (in Level 2), but we
>don't ask for a commitment to review the test suite until level 4.

I would hope that the WGs are already sufficiently motivated to issue
only tests that they "accept" through their internal procedures.

>2) Although we've asked for commitment to the existence of a test suite
>in level 2, we do not ask for test assertions (as am addendum) until
>level 6.

When I originated the set of levels, I knew that I was taking notions
from parallel tracks and trying to make a sensible sequence. In my QA
identity, I want test assertions to be in the table at as low a level
(numerically) as possible. When I think about what has to be "sold to"
the substantive WGs, however, the idea of a set of test assertions
seems (to me) to require quite a lot of convincing. Specifically, it
has a large impact on the information content of the Recommendation
document, and in that sense more impact on their work than the
development of any test suite other than a complete one.

>3) Level 3 asks for the WG to aim to have numerous normative use cases
>in the Rec.  The word "numerous" is vague and unverifiable.

And the way you would set that number might be as a percentage of the
number of test assertions. Along with Mark's idea that the hoped-for
lowest level of commitment might move up to Level Four, this pushes
the topic of test assertions to the forefront.

What reaction will you get if you (in effect) insist that all Recs
ought to have a complete set of test assertions? If that position is
too far advanced beyond current practice, how do you compromise?
Lessen insistence to weaker compulsion?
Sacrifice completeness and ask for them to attempt to be complete?
Insist on differentiation of testable statements from plain text?
.................David Marston

Received on Monday, 11 March 2002 12:13:14 UTC