Final Minutes of 18 August 2003 Teleconference

QA Working Group Teleconference
Monday, 18-August-2003
--
Scribe: Lynne Rosenthal

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(dd) Dimitris Dimitriadis (Ontologicon)
(KD) Karl Dubost (W3C, WG co-chair)
(PF) Peter Fawcett (RealNetworks)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO - WG co-chair)
   (LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
   (VV) Vanitha Venkatraman (Sun Microsystems)

Regrets:
(SM) Sandra Martinez (NIST)

Absent:
(KG) Kirill Gavrylyuk (Microsoft)
(AT) Andrew Thackrah (Open Group)

Summary of New Action Items:
[Format:
AI-20030818-1   KD to review XHTML-Print, Sept 7
AI-20030818-2:  DH to draft text for TestGL on close relation to other
specs  Sept 15

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2003Aug/0053.html
Previous Telcon Minutes:
http://lists.w3.org/Archives/Public/www-qa-wg/2003Aug/0035.html


1.) roll call 11am EDT, membership

2.) Routine Business
- XHTML Print Last Call Review (7-sep):  Karl.

3.) Status of OpsGL transition to CR
The 2 week period for commenters to reply on QAWG's disposition of their
comments closed today. All accepted our dispositions, except 1 (Jon
Gunderson). Proposed way forward 
http://lists.w3.org/Archives/Public/www-qa-wg/2003Aug/0067.html
LH proposed to go forward even though we have one remaining negative.
Unanimous in favor of moving forward and requesting CR transition as per
Part 1 proposal. KD will work with LH to draft letter requesting CR. LH
plans to seek permission to publish document under old-Pub Rules.
4.) TestGL topics
Publish next version  Sept 20.

Overlap between OpsGL and TestGL
It is difficult to determine the boundaries between these documents.  OpsGL
talking about organizational and planning processes, whereas TestGL focus
on implementation, test development and execution.  OpsGL GL5 (plan test
material development) overlaps with what TestGL should talk about.  Another
overlap in TestGL GL7 (planning for conformance testing).   For now, keep
CP5.4  in Ops so not lost and during CR period discuss moving it to
TestGL.  Need to document in the section on relationship to other family
documents, the close relationship between the documents (DH to draft
text).   There is a close relationship and need to draft text to reflect
this. CP5.1 from OpsGL should migrate into TestGL.   Recommendation: if it
fits in TestGL, continue to develop it there. And enter comments during the
OpsGL CR comment period.

Review of Guidelines:
Overview of the guidelines: Analyze spec and determine strategy, Test
Assertions, how to manage, how to execute, document, report, and plan for
testing. Removed ‘test framework’ since this caused much confusion.   All
agreed that this was a good set of guidelines.
GL2 divides metadata into that which is essential and additional
metadata.  CP2.2 “…assertion is ambiguous, contradictory, incomplete,
untestable…” is here since the spec may contain bad requirements and this
identifies/points it out.  Suggest changing wording to where the assertions
identifies ambiguous, contradictory… requirements in the spec.
Assumption is that separate from the spec is a list of assertions and that
these assertions need to be tied back to the spec.- where they come from,
i.e., traceability.
CP3.2  there is a set of metadata to be associated with each test. The
problem is when tests are automatically generated and test assertions are
not created. If you don’t have assertions, then what do you tie this data
to?  Tie it directly to the spec.   Is it viable to have this assertion
metadata associated with the tests or do you need to generate the
assertion? This doesn’t apply solely to auto-generation, that is, tests tie
directly back to the conformance requirements.   There still needs to be
more thought regarding auto-generation of tests and test assertions.