- From: Sandra Martinez <sandra.martinez@nist.gov>
- Date: Tue, 19 Aug 2003 07:50:53 -0400
- To: Patrick Curran <Patrick.Curran@sun.com>
- Cc: QAWG <www-qa-wg@w3.org>
At 05:41 PM 8/17/2003 -0700, Patrick Curran wrote: > >Checkpoint 1.3. Analyze the structure of the specification, partition it >as appropriate, and determine and document the testing approach to be used >for each partition. [Priority 1] > > > >Conformance requirements: The scope, goal, and purpose of the test suite >as a whole, and where appropriate of each logical 'partition' of the test >suite, and the mapping between such partitions and sections of the >specification, must be identified and documented This requirement is not specific to the checkpoint, scope, goal , and purpose of the test suite is covered in CP1.1. The sentence is too long. > >Checkpoint 4.3 Integrate results reporting into the automated test >execution process [Priority 2] > > > >Conformance requirements: If the test execution process is automated, the >automation system must also capture test results. > >Rationale: If tests are executed automatically by some kind of test >harness it will be a relatively simple matter to design the harness to >capture test results for later processing by a results-reporting system. The term harness should also be used in the previous checkpoint. >[ED NOTE: is 4.3 implicit in 4.1 and 4.2?] Both 4.1 and 4.3 are implicit in 4.1, but 4.1 is referring to the execution process and 4.2 and 4.3 are referring to the automation of the process. It is a good idea to keep them separate. > >Checkpoint 6.4. Allow test results to be filtered [Priority 2] > > > >Conformance requirements: The results reporting framework must support >filtering on the same metadata used in the Test Management System. The "reporting framework" terminology is not used in the previous checkpoints. >Rationale: Results should be filterable based on criteria just as test >cases should be. This allows for implementors to focus on tests and the >results of tests that relate to the particular target criteria of the >implementation, while being able to filter out tests and the results of >tests that are not applicable to a given implementation. > Sandra I. Martinez National Institute of Standards and Technology 100 Bureau Drive, Stop 8970, Gaithersburg, Md. 20899 (301) 975-3579 sandra.martinez@nist.gov
Received on Tuesday, 19 August 2003 07:51:24 UTC