LC comment for OpsGL : 'Structure/Organization of Guidelines'

Here is a last call comment from Patrick Curran (Patrick.Curran@sun.com) 
on QA Framework : Operational Guidelines (and Examples and Techniques)
received by the LC form system.

Submitted on behalf of: N/A
Comment type: Substantive
The comment applies to: "Overall"
Comment title: Structure/Organization of Guidelines

Comment:
Comments on the structure and organization of the guidelines and checkpoints of SpecGL.


Overall, I think we have the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. Start with what needs to be done when the WG is formed, and proceed through the spec and test development cycle to the maintenance phase, as we have tried to do with the restructuring of TestGL.


Guideline 1

I find checkpoints 1.1 - 1.3 very confusing. These are "compound checkpoints" that incorporate multiple other checkpoints. We don't use this structure anywhere else in our docs - why here? The document states "This seven-point enumeration is derived from the proposal to the QA mail list, after the 4/2001 QA Workshop", but how we got here is not really interesting to the reader.

The 'sub-checkpoints' on the 'left hand side' of the table are all spec-related. Either these duplicate checkpoints from SpecGL, or they should be incorporated into that document. Those on the 'right hand side' are operations-related, but they seem to overlap with other checkpoints specified in this document.

Recommendation: drop the compound structure, make sure that all the spec-related sub-checkpoints are covered by SpecGL, and move the 'test materials' sub-checkpoints into the body of this doc.

The remaining checkpoints under guideline 1 are different kinds of beasts (not compound) and as such, the transition to them seems somewhat abrupt.


Guideline 2: I'd say "Allocate resources..." rather than "Define resources..."


Guideline 3 begins with some text ("The benefits of....") that seems to be a rationale. Other Guidelines jump straight into the Conformance requirements - this one should too.

How is checkpoint 3.1 different from 1.5?

Checkpoint 3.2 doesn't seem to be directly related to the Guideline under which it's classified. The Note for 3.2 points out that checkpoint 8.2 is related - doesn't this suggest that the overall structure should be re-worked (if they're related, why are they
so far apart numerically?)


Guideline 4

Probably should be #1 (it's the first chronologically). When we summarized this document in our outreach presentation we made checkpoint 4.1 the first bullet item...

Checkpoints 4.1 and 4.2 would seem to belong in Guideline 2 (define/allocate resources) rather than here?

The Discussion for checkpoint 4.3 says "To summarize...". This implies that somewhere there's a more detailed description of what the QAPD must address, but I don't think there is. This checkpoint really seems to amount to "document how you meet these other
checkpoints", yet the list of "other checkpoints" that must be implemented is incomplete. Why doesn't this simply require that *all* checkpoints be documented?

Checkpoint 4.5 uses the ambiguous term "framework" (we tried to avoid this in our re-write of OpsGL).

Checkpoint 4.6 addresses branding - another argument for a chronological rather than a 'logical' grouping (this should be at the end).


Guideline 5

Checkpoint 5.2 - Define a contribution process. Why only priority 2? Without a contribution process you have nothing, surely?


Guideline 6

Checkpoint 6.1 contains a mixture of stuff. The guideline addresses "publication" but much of this checkpoint addresses "management". Moreover, the bullet items in the Discussion
section don't seem to relate to repositories at all.

Checkpoint 6.2 (defining the license for published test materials) is closely related to 5.3 (defining the license for submissions). Should these be grouped together (under a guideline that addresses submission processes)?


Guideline 7

This guideline is labelled "plan the transfer of test materials to W3C if needed", and explicitly states "all of the checkpoints... are not applicable if the WG does not transfer..." (should be "none of the checkpoints are applicable if..."). However, all the
checkpoints seem to apply whether or not the materials are "transferred". It's obviously important to review the quality of submitted tests, to ensure that we have the staffing to deal with submissions, and to resolve IPR issues.


Proposed resolution : 
I think we have approximately the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. 

A possible set of guidelines might be:

Charter 
Allocation of resources
Planning & synchronization
Test submission/development
Test management
Test publication
Conformance testing (test usage)
Maintenance

]]

-- 
This comment was submitted through the lastCall form system,
designed by Martin Duerst and Adapted for the QAWG by Olivier Thereaux.

Received on Friday, 14 March 2003 01:54:44 UTC