Re: Revised Guidelines and Checkpoints for TestGL

At 05:41 PM 8/17/2003 -0700, Patrick Curran wrote:



>
>Checkpoint 1.3. Analyze the structure of the specification, partition it 
>as appropriate, and determine and document the testing approach to be used 
>for each partition. [Priority 1]
>
>
>
>Conformance requirements: The scope, goal, and purpose of the test suite 
>as a whole, and where appropriate of each logical 'partition' of the test 
>suite, and the mapping between such partitions and sections of the 
>specification, must be identified and documented

This requirement is not specific to the checkpoint, scope, goal , and 
purpose of the test suite is covered in CP1.1. The sentence is too long.





>
>Checkpoint 4.3 Integrate results reporting into the automated test 
>execution process [Priority 2]
>
>
>
>Conformance requirements: If the test execution process is automated, the 
>automation system must also capture test results.
>
>Rationale: If tests are executed automatically by some kind of test 
>harness it will be a relatively simple matter to design the harness to 
>capture test results for later processing by a results-reporting system.

The term harness should also be used in the previous checkpoint.


>[ED NOTE: is 4.3 implicit in 4.1 and 4.2?]

Both 4.1 and 4.3 are implicit in 4.1, but 4.1 is referring to the execution 
process and 4.2 and 4.3 are referring to the automation of the process. It 
is a good idea to keep them separate.


>
>Checkpoint 6.4. Allow test results to be filtered [Priority 2]
>
>
>
>Conformance requirements: The results reporting framework must support 
>filtering on the same metadata used in the Test Management System.

The "reporting framework" terminology is not used in the previous checkpoints.


>Rationale: Results should be filterable based on criteria just as test 
>cases should be. This allows for implementors to focus on tests and the 
>results of tests that relate to the particular target criteria of the 
>implementation, while being able to filter out tests and the results of 
>tests that are not applicable to a given implementation.
>

Sandra I. Martinez
National Institute of Standards and Technology
100 Bureau Drive, Stop 8970,
Gaithersburg, Md. 20899

(301) 975-3579
sandra.martinez@nist.gov

Received on Tuesday, 19 August 2003 07:51:24 UTC