LC-59

Ref.  http://www.w3.org/QA/WG/lc-issues#x59

"The following expressions seemed either hard to apply or to test:"

Recognizing that we will always have issues about "testable enough?", here 
are some responses and/or proposals to improve.

LC-59.1
=====
"QA deliverables" is very broad and not defined (GL3)

Discussion.  It has been left intentionally broad, so as to not 
unnecessarily constrain the sorts of QA deliverables.  Right now, the text 
says, "Examples of QA deliverables might range from a TS (test suite) 
production schedule in early WDs, to TS design document in later WDs, to a 
first public TS release at CR."

Proposal.  Would it be an acceptable improvement to replace that with:  "QA 
deliverables include at least the test materials to which the WG has 
committed (see Checkpoints 1.2, 1.3, and 1.4).  For example, the WG might 
synchronize the first public release of a test suite (TS) with Candidate 
Recommendation (CR).  Other examples of QA deliverables might include a TS 
production schedule synchronized with first published WD, a TS design 
document and tests enumeration in later WDs, an interoperability 
matrix/report at beginning of PR, etc.

LC-59.2
=====
"ensure" is not testable in a conformance requirement (at least CP 3.2, 6.1)

Proposal 3.2.  Change,

  "the Working Group MUST ensure that the final published test materials 
support specification versioning/errata, and SHOULD include specification 
versioning/errata considerations in any other QA deliverables such as 
intermediate planning documents."

to:

"the Working Group's final published test materials MUST support 
specification versioning/errata, this MUST be addressed in test materials 
documentation, and the Working Group SHOULD include specification 
versioning/errata considerations in any other QA deliverables such as 
intermediate planning documents."

Proposal 6.1.  Change,

"the Working Group MUST ensure that test materials have repository 
locations that are secure, reliable, and freely accessible."

to,

"the Working Group's test materials MUST reside in repository locations 
that are secure, reliable, and freely accessible."

LC-59.3
=====
"commensurate" isn't either (CP 4.2)

Proposal.  There is the counter-position that commensurate is a good word 
here.  Discussion or OpsET could give the details so that the combination 
is both informative and verifiable.  That notwithstanding, change:

"the Working Group MUST identify a QA task force of size commensurate with 
the QA commitment level and the agreed QA deliverables."

to:

"the Working Group MUST identify and assign a QA task force for the tasks 
necessary to meet the QA commitment level and the committed QA 
deliverables, identified in checkpoints 1.1 - 1.5."  And add to the 
Discussion, "The WG needs to ensure that the size of the task force is 
adequate to meet the committed QA deliverables and QA level, per the 
committed milestones."

[Ed note.  commensurate also appears in CP2.2.  No comments?]

LC-59.4
=====
using the future makes a CP untestable (CP 6.3)

Proposal.  Change:

"in its QA Process Document the Working Group MUST describe how test 
materials will be published, and point to the Web site at which they will 
be published."

to:

"in its QA Process Document the Working Group MUST document the planned Web 
location and method for publication of test materials."

[Ed note.  could also use "intended" instead of "planned".  The point is 
that this is advanced planning, i.e., we want the WG to think about it 
early on, rather than later.  For that reason, one of those words is 
appropriate, and it would change the meaning to drop them.]

LC-59.5
=====
"a quality assessment" is not well defined (CP 7.1)

Discussion.  "a quality assessment" is used in the statement of the 
checkpoint.  It does not have to be testable there.  Does Originator still 
have problems with "perform and record an assessment of the quality of the 
test materials," in the Conformance Requirements?  If so, what is the 
nature of the problem?

Any sort of quality assessment is going to necessarily be pretty 
TM-specific, depending the scope & goals of the TM, the type of TM (think 
"taxonomy"), etc.  Is some sort of generic list desired, such 
as:  "including at least correctness, traceability, atomicity, user 
documentation, maintainer documentation, declaration of scope, completeness 
(vis a' vis declared scope), harnesses or interfaces for application of the 
TM, configurability, results assessment, results recording & reporting, 
automation features, versioning/errata support, declaration of publication 
licenses,   integrated submission procedures, etc."

Another thought:  all of this stuff could be in SpecET, in a template or 
checklist.

Proposal.  Originator propose wording that is satisfactory.

LC-59.6
=====
"sufficient" is not testable (CP 7.2)

Discussion.  "sufficient" is used in the statement of the CP, which is its 
title or label.  The CP statement need not be testable, but rather should 
indicate in clear language what is the subject of the CP.  Presumably, 
Originator would also dislike "commensurate" in the Conformance 
Requirements, similar to the CP4.2 comment.

Proposal.  Change,

"as a part of any test materials transfer process, the Working Group MUST 
commit staff resources commensurate with planned ongoing WG test materials' 
deliverables, and MUST record that commitment in some consensus WG document."

to

"as a part of any test materials transfer process, the Working Group MUST 
identify and assign staff resources for the tasks associated with ongoing 
WG test materials' development and maintenance, and MUST record that 
commitment in some consensus WG document."
And add to the Discussion, "The WG needs to take care that the amount of 
staff resources is adequate to meet needs of ongoing development and 
maintenance of the transferred test materials."

LC-59.7
=====
'all' is not testable (CP 7.3)

Proposal.  Delete the occurrence of "all" in Conformance Requirements.
### end ###

Regards,
-Lofton.

Received on Thursday, 8 May 2003 21:29:04 UTC