LC-60.10, 60.13, 60.15

This is the THIRD and final piece of several pieces that collectively 
propose *specific* resolutions for LC-60.3 through LC-60.15.

First piece: LC-60.3, 60.5, 60.7, 60.8
http://lists.w3.org/Archives/Public/www-qa/2003May/0028.html

Second piece: LC-60.4, 60.6, 60.9, 60.11, 60.12, 60.14
http://lists.w3.org/Archives/Public/www-qa/2003May/0033.html

This third piece deals with: LC-60.10, 60.13, 60.15

See [4] for illustration of the implementation of the proposed resolutions.

Overview
-----
For resolutions of LC-60.3 through LC-60.15, a proposal for resolutions was 
circulated a couple weeks ago at:
[1] http://lists.w3.org/Archives/Public/www-qa/2003Apr/0064.html

Since then, we have resolved LC-60.1 to "clarify and solve the particular 
problems, but try to avoid major re-org".

Proposed resolutions follow. Discussion is welcome on this list.


LC-60.10:
=====
Checkpoint 4.5 uses the ambiguous term "framework" (we tried to avoid this 
in our re-write of OpsGL).

Proposal:  LC-72.7 [5] also raises this issue.  Resolved LC-72.8 suggests 
P2 instead of P1 (accepted).  Other comments indicate confusion why CP4.5 
is here at all, instead of TestGL.  All of these are addressed in the 
following proposal, for complete replacement text:

===== complete current text =====
CP4.5 Define the QA framework for test materials development. [Priority 1]

Conformance requirements: the WG's QA Process Document MUST define the 
framework that will be used for test materials development.

Rationale. The framework is effectively the top-level design for test 
materials production. As in software development projects, a sound 
top-level design enables the efficient production of quality deliverables 
that work as intended.

Discussion. The QA framework describes how to develop, document and use the 
tests.
===== end current =====

===== proposal =====
CP4.5 Define a framework for test materials development. [Priority 2]

Conformance requirements: the WG's QA Process Document MUST define a 
framework for test materials development.

Rationale. A test materials development framework is effectively the 
top-level design for test materials production. A sound top-level design 
enables the accurate allocation and application of resources -- staffing 
and logistical -- and the efficient production of quality deliverables that 
work as intended.

Discussion. While consideration of a test materials framework might seem to 
be a detail that belongs only in "QA Framework: Test Guidelines", in fact 
such a framework has potential impacts on a WG's operations, processes, 
staffing, and logistics, and therefore its consideration in the QA Process 
Document is appropriate.
===== end proposal =====


LC-60.13:
=====
Guideline 6: Checkpoint 6.1 contains a mixture of stuff. The guideline 
addresses "publication" but much of this checkpoint addresses "management". 
Moreover, the bullet items in the Discussion section don't seem to relate 
to repositories at all.

Discussion.  Good points, especially about the bullet items.  About 
"management", one could argue that some of the things we discuss un GL6, 
like repository considerations, are related to (and pre-requisite to) 
publishing.

Proposal:  Take the bullet list out of CP6.1, fine tune the wording, and 
put it in the verbiage of GL6.  Add comment to the GL6 verbiage explaining 
the relationship of some management aspects to publishing, and therefore 
the treatment of those here in GL6.

Proposed new verbiage for GL6:
=== start ===
"Once the test materials (TM) development is in progress, a Working Group 
needs to publish the TM drafts and releases, as part of its QA 
processes.  When the test materials have reached some advanced milestone of 
maturity and development (e.g., operationally usable), the WG needs to 
ensure that:

** their support and maintenance includes secure repository;
** tests materials are freely available for download;
** any licenses do not preclude the test suite from being used by all 
interested parties;
** publication of test results is encouraged.

Meeting the needs of TM publication necessarily involves some aspects of TM 
management, such as repository.  These prerequisite aspects are addressed 
in the following checkpoints, as well as publication details proper."
=== end ===

LC-60.15:
=====
Guideline 7: This guideline is labelled "plan the transfer of test 
materials to W3C if needed", and explicitly states "all of the 
checkpoints... are not applicable if the WG does not transfer..." (should 
be "none of the checkpoints are applicable if..."). However, all the 
checkpoints seem to apply whether or not the materials are "transferred". 
It's obviously important to review the quality of submitted tests, to 
ensure that we have the staffing to deal with submissions, and to resolve 
IPR issues.

Discussion.  All of the submission stuff gets dealt with in GL5.  GL7 is 
about the wholesale transfer of an entire externally written test 
suite.  Granted, this could be considered as a "submission", but the sense 
of GL5 is I think more piecemeal submission during the building of a 
suite.  We (QA) have had experience moderating such a transfer, which is 
why GL7 exists -- it's useful to have it in one place, even if there are 
parallels elsewhere in OpsGL for piecemeal planning, development, staffing, 
submission, etc.

Proposal:  Fine tune the wording of the GL7 verbiage, and insert this new 
2nd paragraph of rationale/explanation:

"It will be noticed that some aspects of the transfer of a whole test suite 
from outside W3C parallel aspects of in-house planning and development, and 
piecemeal contribution and review.  Those considering such transfers might 
not already have the planning, development, and submission infrastructure 
in place. Previous transfer experiences have shown that it is useful to 
isolate and collect those aspects in one place, as this guideline does."
### end ###

Regards,
Lofton.

[2] http://www.w3.org/QA/WG/lc-issues#x60
[3] http://www.w3.org/TR/2003/WD-qaframe-ops-20030210/
[4] http://www.w3.org/QA/WG/2003/05/qaframe-ops-20030514
[5] http://www.w3.org/QA/WG/2003/03/qareview20030314.html#OG-7