Overlap between OpsGL and TestGL

It's sometimes difficult to draw the boundaries between TestGL and OpsGL. In critquing TestGL we've said that it must address how tests are developed and submitted yet this is covered in OpsGL GL5 (see next issue). We said we must address licensing, but this is covered in OpsGL 5.3 and 6.2. We said we must address versioning, but that's covered in OpsGL 8.2.

Many of the topics suggested for addition to TestGL (see annotated Crete minutes) are in fact covered in OpsGL.

One possible approach to this problem is...

OpsGL addresses planning, organizational, and communication processes, while TestGL focuses more on implementation, and on the technical aspects of test development and execution.

This division works pretty well for OpsGL's Guideline 6 ("Plan test materials publication"), which focuses on the logistics of publication. TestGL could therefore address the need to test and document the test suite before publication.

The division doesn't work so well for OpsGL's Guideline 5 ("Plan test materials development"). This should focus on the process for soliciting test materials, and for reviewing them (including providing feedback to submitters), and should leave the technical details to TestGL.

OpsGL GL 5, has four checkpoints:

5.1. Ensure test materials are documented and usable for their intended purposes.
5.2. Define a contribution process.
5.3. Address license terms for submitted test materials.
5.4. Define review procedures for submitted test materials.

The first of these checkpoints should probably be moved to TestGL, where it could be expanded into a Guideline addressing the need to test and document the submitted materials.

5.1 only addresses these issues very superficially. (Interestingly enough, the conformance requirements and discussion address
documentation, but not whether the materials are "usable" (which could be considered code for 'test the test materials').

I therefore added a new Guideline 4 to TestGL, addressing the need to test and document the test materials, and I recommend removing 5.4 from OpsGL.

Note also that Guideline 7 from this version of TestGL might be better positioned in OpsGL.

Other Issues

Need a Concepts section?

Do we need more discuss of test materials-management, test execution, and results reporting, and the overlaps between them or are these terms now sufficiently clear in the text?

For each, the process must be well defined. Automation is encouraged, but not required. If automation is introduced, the systems may be integrated.

NOTE: The terms are introduced before they are defined in their respective Guidelines, so at the very least we'll need forward-references.

We could also discuss Metadata, and its uses in classifying, filtering, sorting.

Only one checkpoint on coverage. Is this sufficient?

Address differing strategies at different points in the development cycle

Lynn's concern. (Also, indirectly, Jeremy Carroll's re 'waterfall model.) Early in cycle may want to devise tests specifically to test the spec itself. Middle of cycle, traditional conformance tests. Later, interoperability...

Issues from Crete F2F discussion not yet addressed

Review of Introduction
Disagreement on whether a scope can be a set of requrements. Class of
products also included interoperability. Intended audience, delete
conformance. Remove or move last paragraph of 1.4. Sections 1.5, 1.6, 1.7
make sure that wording is consistent with other documents. Need to create
use cases for TestGL.

Review of Conformance and Definitions
Definitions need work. Conformance Section needs to be reviewed.