W3C home > Mailing lists > Public > www-qa-wg@w3.org > August 2003

Re: Revised Guidelines and Checkpoints for TestGL

From: Sandra Martinez <sandra.martinez@nist.gov>
Date: Tue, 19 Aug 2003 07:50:53 -0400
Message-Id: <>
To: Patrick Curran <Patrick.Curran@sun.com>
Cc: QAWG <www-qa-wg@w3.org>

At 05:41 PM 8/17/2003 -0700, Patrick Curran wrote:

>Checkpoint 1.3. Analyze the structure of the specification, partition it 
>as appropriate, and determine and document the testing approach to be used 
>for each partition. [Priority 1]
>Conformance requirements: The scope, goal, and purpose of the test suite 
>as a whole, and where appropriate of each logical 'partition' of the test 
>suite, and the mapping between such partitions and sections of the 
>specification, must be identified and documented

This requirement is not specific to the checkpoint, scope, goal , and 
purpose of the test suite is covered in CP1.1. The sentence is too long.

>Checkpoint 4.3 Integrate results reporting into the automated test 
>execution process [Priority 2]
>Conformance requirements: If the test execution process is automated, the 
>automation system must also capture test results.
>Rationale: If tests are executed automatically by some kind of test 
>harness it will be a relatively simple matter to design the harness to 
>capture test results for later processing by a results-reporting system.

The term harness should also be used in the previous checkpoint.

>[ED NOTE: is 4.3 implicit in 4.1 and 4.2?]

Both 4.1 and 4.3 are implicit in 4.1, but 4.1 is referring to the execution 
process and 4.2 and 4.3 are referring to the automation of the process. It 
is a good idea to keep them separate.

>Checkpoint 6.4. Allow test results to be filtered [Priority 2]
>Conformance requirements: The results reporting framework must support 
>filtering on the same metadata used in the Test Management System.

The "reporting framework" terminology is not used in the previous checkpoints.

>Rationale: Results should be filterable based on criteria just as test 
>cases should be. This allows for implementors to focus on tests and the 
>results of tests that relate to the particular target criteria of the 
>implementation, while being able to filter out tests and the results of 
>tests that are not applicable to a given implementation.

Sandra I. Martinez
National Institute of Standards and Technology
100 Bureau Drive, Stop 8970,
Gaithersburg, Md. 20899

(301) 975-3579
Received on Tuesday, 19 August 2003 07:51:24 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:31 UTC