DRAFT MINUTES 02-08-02 QAWG Teleconference

DRAFT MINUTES
QA Working Group Teleconference
Friday, 8-March-2002

Scribe: Jack Morrison 

Attendees:
(DD) Daniel Dardailler (W3C - IG co-chair)
(KD) Karl Dubost (W3C, WG co-chair)
(KG) Kirill Gavrylyuk (Microsoft)
(LH) Lofton Henderson (CGMO - WG co-chair)
(JM) Jack Morrison (Sun)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(OT) Olivier Thereaux (W3C - systems) 

Regrets: 
(PF) Peter Fawcett (Real Networks)
(DH) Dominique Hazael-Massieux (W3C - Webmaster)
(AT) Andrew Thackrah (Open Group)

Absent: 
(DD) Dimitris Dimitriadis (Ontologicon)
(KH) Katie Haritos-Shea (DOC)

Summary of New Action Items: 
A-2002-03-08-1 - OT & KD – update web pages to reflect new name
A-2002-03-08-2 - KD – Inform director of name change
A-2002-03-08-3 - KG – Definition of priorities to be added to Operational Guidelines
A-2002-03-08-4 – KD – investigate impact of guideline related to call for participation on the overall W3C process
A-2002-03-08-5 – KG & LH – Clarification of Guideline Checkpoints

Previous Telcon Minutes: http://lists.w3.org/Archives/Public/www-qa/2002Feb/0016.html

Plenary F2F Minutes: http://www.w3.org/QA/2002/03/01-f2f-minutes  

Minutes: 

LH opened the meeting with an explanation of the objective, which was to work on Issue #55 - Checkpoint Priorities in Proc&Ops 
need to be confirmed. He also mentioned that the first WG-only drafts of the Specifications Guidelines and Operational Examples and 
Techniques were to be released next week.  This led to a discussion about resolving the name of the Process and Operational 
Guidelines document. It was felt that some good discussion had taken place on which name it should be, Operational Guidelines or 
Process Guidelines. A roll-call vote was taken on the two choices:

Daniel Dardailler - Operational Guidelines
Karl Dubost - Operational Guidelines
Kirill Gavrylyuk - Process Guidelines
Lofton Henderson - Operational Guidelines
Jack Morrison - Process Guidelines
Lynne Rosenthal - Operational Guidelines
Mark Skall - Operational Guidelines
Olivier Thereaux - Operational Guidelines

It was agreed that the document be renamed QA Framework: Operational Guidelines.

The rest of the meeting dealt with Issue #55, Checkpoint Priority. The only comments that had been distributed by email were from 
LH, so the discussion started with a review of his comments. 

LH Comment 1 was related to the criteria used to set priority. LH felt that currently the use of priority values was arbitrary, as there 
was no definition. Most members agreed that this was true and that some definition was necessary. The use of MUST SHOULD MAY 
was discussed, as was the verbiage utilized by WAI. It was agreed that something needed to be added to the specification to define 
how priority was arrived at, but that that was a long discussion itself and more difficult that what was defined in the WAI document. It 
was also agreed that one of the things we were trying to achieve with the WG effort was defining a process that would cause good test 
suites to be generated but make sure that they be done early enough to be useful. This lead to the comment that the definition of 
priority must be oriented toward the goal of high quality, timely production of test materials. KG agreed to develop a draft set of 
definitions to be included in the Operational Guidelines, have LH review them and then distribute them to the rest of the group for 
review. The question of how to rank things today was discussed and it was agreed that high quality, timely production of test materials 
should be kept in mind.

LH Comment 2 - After the Technical Plenary it was obvious that there was a universal issue related to resources. Given that, LH felt 
that Guideline 2.2 – In charter, address QA staffing requirements, should be changed from Priority 2 to 1. There were no objections.

LH Comment 3 – Related to comment 2, there was a long discussion about whether a new checkpoint should be added that mentioned 
specifically recruiting QA people during the Call to Participation phase. The overall ramifications of this on the W3C process were 
discussed, as the CFP was usually distributed by the chair/co-chair. It was a concern that if we wanted to change the definition of the 
Activity Phase that we would need to change the W3C process, but that if it was part of the charter then it was up to the individual 
working group. It was also mentioned that there were two parts to the resource issue; 1) defining the resource requirements, which we 
already require, and 2) obligating people to provide resource, which we do not. It was agreed that, for now, a new guideline, 2.3, 
would be added to stating that the Call for Participation needed to clearly indicate that members needed to contribute QA resources. 
KD said he would look into the CFP issue from a W3C perspective.

LH Comments 4 & 5 – Normative language was discussed a lot during the discussion on Comment 1.

LH Comment 6 – The wording on guidelines 1.3 and 3.1 make it unclear what the difference is. A couple of members mentioned this 
and LH agreed to re-write the definitions and distribute them via email for comments.

The remainder of the meeting was spent going through each of the Guidelines and discussing the priority.

Guideline 1. Integrate Quality Assurance into Working Group activities.
1.1 In Charter, determine level of commitment to QA. Plan to have at least a commitment to "QA level three".
[Priority 1]
It was discussed whether Level 3 was the correct level for this checkpoint, as the definition of the levels changes. It was agreed that 
more time was required for review of this item, but that for now it would stay P1.

1.2 In Charter, identify QA deliverables, expected milestones
[Priority 2] -> [Priority 1]
It was discussed and agreed to that to ensure “high quality, timely production of test materials” that this should be a P1.

1.3 In Charter, identify QA criteria for WG deliverables milestones.
[Priority 2]
KG & LH to work on new wording for this section so that it is clearly differentiated from 3.1, but it is still a P2.
   
Guideline 2. Define resources for Working Group QA activities.

2.1 In Charter, address where and how conformance test materials will be produced.
[Priority 1]
No change.

2.2 In charter, address QA staffing commitments.
[Priority 2] -> [Priority 1]
Given the previous discussion and concern about getting resources committed, this was move to P1.

2.3 As part of Call for Participation, define the need for members to contribute QA resources
[Priority P2]
This is a new guideline. KD to investigate it from the W3C process perspective. KG to add to draft for Monday.

Guideline 3. Synchronize QA activities with the milestones for WG deliverables.

3.1 Align intermediate milestones for QA deliverables with WG deliverables milestones
[Priority 2] -> [Priority 3]
Discussed what this guideline meant and how it related to 1.3. KG & LH to reword it so it is clear in relationship to 1.3. Also agreed, 
after the discussion, that as reworded it should be a P3.

3.2 Synchronize publication of QA deliverables and specification's drafts.
[Priority 2]
Initially it was thought this might be included in 3.1, however there was not enough description on what it meant and a number of 
different perspectives were presented. Needs clarification that it has to do with Public distribution, but remains a P2.

3.3 Support specification versioning/errata in the QA deliverables.
[Priority 3]
 Discussed if this belonged here or in Guideline 8. Needs to be made clearer, but can stay as a P3 for now.

Guideline 4. Define the QA process.

4.1 Define the goals and scope of QA development
[Priority 1]
Discussed that this includes goals and non-goals and is similar to 1.1. Might want to expand 1.1 and use this as an example. KG to 
think about how it fits and update the specification for Monday.

4.2 Produce the QA Process document.
[Priority 1]
Discussed this for a while, but did came to no conclusion. KG asked everyone to review it and make sure it made sense.

 LH discussed the need for an extra meeting the week of 3/18. It was agreed we would decide at next Thursday’s meeting if the 
meeting was necessary. LH proposed we split the Thursday meeting into 1) a discussion of the progression of the new documents that 
are coming online 3/11, and 2) a continuation of the discussion of Issue #55. LH also indicated that #55 was a higher priority, as it 
needs to be done as soon as possible.

Next Meeting
Thursday, March 14, 2002
2PM EST

Meeting adjourned at 3:30PM EST.