[Draft] Minutes of QA WG telecon 2003-12-22

QA Working Group Teleconference
Monday, 22-December-2003
--
Scribe: Lofton Henderson

Attendees:
(KD) Karl Dubost (W3C, WG co-chair)
(LH) Lofton Henderson (CGMO - WG co-chair)
(SM) Sandra Martinez (NIST)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)

Visitor:
(DM) David Marston

Regrets:
(DD) Dimitris Dimitriadis (Ontologicon)
(KG) Martin Chamberlain (Microsoft)
(DH) Dominique Hazaël-Massieux (W3C)
(AT) Andrew Thackrah (Open Group)
(VV) Vanitha Venkatraman (Sun Microsystems)

Absent:
none.

Summary of New Action Items:
No new action items (except informal "start email discussion on [topic]" -- 
see minutes below.)

Agenda:
http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0056.html

Previous Telcon Minutes:
http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0047.html [DRAFT]

Minutes:

Agenda item:

>2.) TA list for SpecGL
>         - cover msg [3a] & list for 12-sept WD [3b]
>         - issues:  [4a], [4b]
>
>[3a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0018.html
>[3b] 
>http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/att-0018/Assertions.html
>[4a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0032.html
>[4b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0034.html

Plus additional issues identified in email:
[5] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0061.html

CP-by-CP issue discussion:

[
CP Number: 2.4

Issue:  Should ConfReqs have non-applicability sentence?  Or should they 
use a preceding "If..." clause?

Discussion: Discussed at previous telecon also.  Question seems to be 
mostly about the form of the applicability exemption -- preceding "If" 
clause versus following non-applicability sentence.  The latter was 
informally adopted as clearer, and a compromise, during discussion of Last 
Call issue from Ian Jacobs (who proposed formal "normative exclusion" 
section in every CP).  No agreement.

Also it was asserted and debated whether being consistent would result in 
applicability exemption on almost every CP.  No agreement.

Decision: deferred
]
[
CP Number: 2.4

Issue:  Same as previous issue, except for TAs derived from ConfReqs.

Discussion: Similar.  No agreements.

Decision: deferred
]
[
CP Number: 3.1

Issue:  Two parts:  How to parse a complicated TA like, "EITHER a1 OR a2 
AND a3" [is it "(EITHER a1 OR a2) AND a3", or "EITHER a1 OR (a2 AND a3)"? 
And, why are we compositing separate simple ConfReqs into single complex TAs?

Discussion:  MS said intent was that parentheses would be used to clarify, 
and they where unintentionally omitted here.  No one advanced good 
rationale for compositing simple ConfReqs.

(Side issue arose:  why generate TAs instead of ConfReqs?  I.e., suitably 
rigorous and clearly identified ConfReqs could fulfill the role of TAs in 
test materials production.)

Decision: parenthesize where ambiguous; keep TAs factored per ConfReq; 
(side issue deferred, see some discussion at end of minutes)
]
[
CP Number: 3.2

Issue:  The ConfReqs of this CP cross reference the results of another CP, 
"@@each identified CoP@@". It is not clear what the reference means (is it 
informative "related checkpoints", or does it establish a normative 
dependence of this CP on the other one?)

Discussion:  This issue is really about the ConfReqs, although the 
identical phrasing, "each identified CoP", is carried directly into the 
TA.  It is also similar to an issue discussed in previous telecon, whose 
conclusion was:  wherever it is easy to do, try to eliminate cross 
dependencies amongst CPs' ConfReqs and TAs.  This could be done here by 
changing the linked phrase "@@each identified CoP@@" to something like 
"each CoP for which the specification defines conformance requirements".

Issue is not clear to everyone.

Decision:  LH to summarize and continue in email.
]
[
CP Number: 3.4

Issue:  use of "any" vs. "all"; and, why is bullet list in ConfReqs but not 
in TAs?

Discussion:  How do you know that the section (required in the TA) has 
identified "all"?  In context of spec, you can only know "all in the spec 
(somewhere)", and that is what "all" means here.  Outside of context of 
spec -- meaningless to talk about "all".

MS pointed out a deeper issue here:  the ConfReqs/TAs are untestable as 
written.  After some discussion, general agreement that ConfReqs (& TAs) 
needed to be changed to something like, "either indicate that all modules 
are mutually independent, or describe the interrelationships and 
dependencies".

Also agreed that the bullet list in ConfReqs belonged in exemplary 
"Discussion", not in the ConfReqs (and TAs).

DM explained another related issue [...scribe missed this...]

Decision:  both "all" and "any" are problematic when used in open-ended 
conformance requirements, but they are okay in the context of the subject 
specification; move bullets into Discussion;  change ConfReqs along the 
lines of, "spec must either indicate that there are no interrelationships 
or dependencies amongst modules, or enumerate the interrelationships or 
dependencies."  (Exact wording to be worked out by editors.
]
[
CP Number: 3.5

Issue:  The wording of CP statement especially, and ConfReqs to a lesser 
extent, confuses whether the meaning is "relationship of p/m/l with other 
DoV" versus "p/m/l amongst themselves and with other DoV"

Discussion:    The latter is intended.

Decision:  break the single composite p/m/l ConfReq into three separate 
ConfReqs (bullets), one each for p & m & l.
]
[
CP Number: 4.1

Issue:  The ConfReq is ambiguous about the "in one place" requirement that 
is implied by the Rationale, and that was synthesized into the TA.

Discussion:   The "in one place" is intended as a requirement.  And it is 
not covered by the CPs of GL7 and GL8, that require fast way to find key 
conformance information.

Decision:  Change the ConfReq along these lines.  Currently:  "The 
specification MUST document in a @@normative@@ section each @@deprecated 
feature@@".  [Matching double-@ indicate link.]  Change to "The 
specification MUST @@list@@ in a single @@normative@@ section each 
@@deprecated feature@@".  And @@list@@ will link to a definition that makes 
clear that the listing can either be a list of links to detailed 
definitions in the body of the spec, or can in fact be a list of detailed 
descriptions of the deprecated features.
]
[
CP Number: 4.6

Issue:  listing of obsolete features should be handled similar to 
deprecated features.

Discussion:

Decision:  agreed, the ConfReqs of CP4.6 should be written in parallel 
fashion to CP4.1.
]
[
CP Number: 5.2

Issue:  DM raised issue about "obviousness".

Discussion:  "Allowable differences" could go arbitrarily deep and is 
potentially an open-ended (and unimplementable) requirement.  No 
suggestions how to do it, but some threshold of directness or obviousness 
would be nice, to limit the open-ended-ness.

Decision:  deferred to email.  (DM will launch email discussion.)
]

Out of time for per-CP TA review.

MS points out that TAs have been very useful in flushing out problems with 
the spec.  PC thinks that the ConfReqs can be
tightened to the point of making separate TA list unnecessary.  DM -- why 
not get rid of ConfReqs and write with TAs.  MS/LH -- that suggestion is 
contra common practice, plus we have confirmed the value of the TA 
derivation on the quality of the ConfReqs -- without requiring the TA list, 
there is not testable evidence that the process (ConfReqs 
quality/testability review) has been done.

KD made a related point, about how to pursue the last topic -- when we 
discuss it, it should not be about should we or should we not put test 
assertions in the specs? But more look at and define use case scenarios wrt 
editors.  So an editor has to write test assertions, does he/she have a 
template to do it?  What are the benefits, the markup?   An XSLT to produce 
them, a tool to easily associate conf req and test assertions, etc.

KD will launch an email discussion about his suggestions.

We decided to pick the TA topic up again on 12th Jan.  Postpone new-TestGL 
review to 14th (Wed), and cancel
following week.  That's okay with all of today's attendees.  LH to query 
QAWG to see if anyone else objects.

Adjourned about 12:10pm EST.
### end ###

Received on Monday, 22 December 2003 19:56:33 UTC