draft minutes of QA F2F meeting in Boulder, 22 Oct PM

Forwarded for Sandra

October 22, 2003 (afternoon) Working draft minutes:

AI-20031022-7: come up with the definition for the term testable. Due October
29.

TestGL discussion continues (concept section):

It was suggested that the concept section should also include the type of test
materials that can be tested. It was decided to point to the class of products
listed in section 2.2 in the SpecGl instead of rewriting to avoid duplication.
Alex brought up the issue that at sometime we need to define what is testable
and was given the action to come up with the definition.

Guidelines

Guideline 1.

Change title to: Define testing strategy.

Checkpoint 1.1.

Remove the word goal from the conformance requirement.

The discussion section should include examples and everything else that do not
belong in the rationale. The rationale most be kept concise. Remove the example
cited in the rationale and add the HTTP (client only, server only) example
provided by Alex.

Checkpoint 1.2.

Rewrite conformance requirement to: Test documentation, must identify the
specification to be tested.
In the rationale use the term “explicitly tests other specifications”.

Checkpoint 1.3.

Change title of this checkpoint to:  Define a testing approach.

The term “testing approach” must be defined (refers to what kind of test is
going to be developed) and provide examples (Validators, API testing,
Protocol). Also, include partitioning as part of the discussion section.

Rewrite the conformance requirement to: A testing approach must be identified
as a result of the specification analysis.

Guideline 2.

Checkpoint 2.1.

Rewrite checkpoint to: Provide a list of test assertions.

Change to priority 1.
Add some verbiage to the discussion to include examples of derived assertions.
Checkpoint 2.2.

The discussion for this checkpoint should include more detail including the
possibility of extracting more than one assertion from the same quoted text. It
is up to the test developer to decide the mechanism for assigning a unique
assertion-ID.  The metadata in this checkpoint should be tied with the metadata
in checkpoint 3.2.

Replace location for the text from which the assertion was derived.

Guideline 3.

The title of this guideline should be revised.

Suggested title: Support test material metadata.

Checkpoint 3.1.

Drop checkpoint, it belongs in OpsGl. TestGl should not address working groups.

Checkpoint 3.2.

Include a statement clarifying that test materials are not referring to
testcases.

In the conformance requirement the third bullet should go back to assertions.
The fifth bullet is too ambiguous, it must be clarified.

In the discussion section:

 The third bullet should go back to assertion as an optional metadata.
 Fourth bullet should also go back to assertions but as mandated.
 Fifth bullet should go back to assertion but rewording conformance level to
degree of conformance.
 A new bullet should be added to include conditional tests as an optional
metadata.

Checkpoint 3.3

This checkpoint should stay but the conformance requirement should be modified
to publish the list as well as the percentage.

Checkpoint 3.4

Drop this checkpoint.

Checkpoint 3.5

This checkpoint relates to the metadata. The checkpoint as well as the
conformance requirement   should be reworded to reflect this relationship. Do
not use the word “process”.

Guideline 4.
Checkpoint 4.1.
Re-write conformance requirement to: The process for executing tests must be
well defined and must document how to execute the test.
A checkpoint should be added for test results reproducibility and repeatability
(test results must be the same regardless who execute the tests – “the same
stuff will happened”
Karl captured the explanation of these two concepts to help define the
checkpoint.



----- End forwarded message -----



----- End forwarded message -----

Received on Thursday, 23 October 2003 14:19:25 UTC