questions/comments on TSDTF test process document

Thanks for the great work on the test process document [1].

Questions (1) and (2) are on the "checklist for content review" of the test 
process document:

(1) What is an "unambiguous unit"?  Does this mean that the test sample and 
test procedure
are consistent?  If so, wouldn't that point be covered in the point previous?

(2) Should it be stated what the expected output of the test is, exactly 
how successful
accomplishment of the test case satisfies the associated success criterion, 
the test
purpose (if not already stated), and if the expected result is not 
achieved, what can
be done to correct the test case so that it will pass (using testing as a 
teaching tool)?
(NOTE: Perhaps some/all of this is covered in the associated metadata for 
the test sample.
Just checking to make sure that the items under Question 7 - "what makes a 
good test? in Quality Assurance
Test Frequently Asked Questions [2] are at least considered in this or 
other parts of the document..)

Questions (3) and (4) are on general review process:

(3) Should the review process explicitly mention the possibility that the 
result of a review may
be a suggestion to correct the associated technique, or even success 
criteria.   In other words,
if there is a "problem" perceived with a submitted test sample, maybe the 
associated specification
or success criterion/technique is in "error", not necessary the test sample 
- should this possibility
be considered?

(4) Is there a time limit on the review process, and notification of the 
submitter of the results of the
review?

If these points are already covered, and I missed them, please accept my 
apologies.

Thanks and happy holidays
Tim Boland NIST

[1]: http://www.w3.org/WAI/ER/2006/tests/process
[2]: http://www.w3.org/QA/WG/2005/01/test-faq#good

Received on Thursday, 14 December 2006 14:43:57 UTC