Draft minutes of QA WG meeting 2003-09-08

Please review by Friday EOB.

QA Working Group Teleconference
Monday, 08-September-2003
--
Scribe: Dominique Hazaël-Massieux

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(SM) Sandra Martinez (NIST)
(LR) Lynne Rosenthal (NIST - IG co-chair)

Regrets: 
(dd) Dimitris Dimitriadis (Ontologicon)
(LH) Lofton Henderson (CGMO - WG co-chair)
(VV) Vanitha Venkatraman (Sun Microsystems)

Absent: 
(PF) Peter Fawcett (RealNetworks)
(KG) Kirill Gavrylyuk (Microsoft)
(MS) Mark Skall (NIST)
(AT) Andrew Thackrah (Open Group)

Summary of New Action Items: 
AI-20030908-01 Karl to establish a preliminary list of WGs, contact them
(and possibly let the QA WG know) for OpsGL Implementation Plan by
2003:09-09
AI-20030908-02 DM to work with Patrick and Lynne on integrating SpecGL
CP 5.4 in 5.5 or 5.1 by 2003-09-10 EOB

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0027.html
Previous Telcon Minutes:
http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0007.html

Minutes: 
-------------------
Status of OpsGL CR
KD informed the WG that OpsGL got Director's approval to go to Candidate
Recommendation, provided that the QA WG puts up a implementation plan
with commitments from other WG to implement (at least partially) the GL.
The goal is to have each CP implemented at least twice.
Patrick suggested to contact in priority the WG we already know are
commited to a QA process, esp. those that we quotes are good examples in
OpsET (DOM, XML Core and SVG). In addition, DH reminded that some WG
showed some interest during the outreach done in the last Technical
Plenary; we could contact e.g. the WS Description WG. KD also thinks the
WAI WG could have interest in that. As test-oriented WG, the OWL WG
could also be a good target.
-> AI-20030908-01 Karl to establish a preliminary list of WGs, contact
them (and possibly let the QA WG know) for OpsGL Implementation Plan by
2003:09-09
PC pointed that this emphasizes the need that we get other WGs more
involved in the GL we develop (need that was stressed for TestGL
recently).


-------------------
SpecGL Topics: status, todo and plans
Dom reminded that Mark sent the Test Assertions list derived from the
latest SpecGL editorial version [1] and asked the WG members to review
it for the upcoming publication.
Then, the discussion went on CP 5.4 future, as per the latest discussion
on the mailing list [2]: what should be done with this complex CP? Keep
it (and wait for feedback from the issues originators), replace it by
another rewrite (volunteers?) or drop it altogether?
Lynne had difficulties understanding the formulation of the CP (although
DM's latest rewrite made it clearer); DM pointed that the original
intent of the CP that Lynne had was about deterministic behavior for
discretinary items; although this could be turned in a new CP, DM's idea
concerned the interaction of multiple independant discretinary items,
which imply a combinatory number of conforming and not-interoperable
implementations, quoting the specific case of the 'min' function in the
current XPath Functions & Op WD.
With this example, PC understood what the intent of the CP was, and
proposed to reformulate it based on justifying the number of independant
discretionary items and giving the issue of combinatory number of
conforming behavior as rationale. DH proposed that such a formulation
could be integrated either in CP 5.1 (justify discretionary items) or
5.5 (document interaction between DoV and discretionary items). It was
proposed that such a rewrite be attempted before the next publication of
SpecGL on Friday, on a very tight schedule.
-> AI-20030908-02 DM to work with Patrick and Lynne on integrating
SpecGL CP 5.4 in 5.5 or 5.1 by 2003-09-10 EOB

1. http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0018 & 
http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/att-0018/Assertions.html
2. http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/thread.html#13

-------------------
TestGL topics, lead by Patrick
Patrick wanted to raise again the question of the importance we attach
to the test assertions list, along to determining the minimal set of
metadata that should be bound to these Test Assertions (with the related
question of knowing whether they should be attached to the Test
Assertions or the Test Cases).
Patrick's opinion was that a list of identified test assertions was
absolutely necessary, if only to justify the results of the tests cases
- e.g., if an implementation fails to pass a test, it must be easy to
say whether the test case is wrong (bad derivation from the test
assertion) or if the implementation is. He noted that this could also be
achieved by setting the statement of purpose as a mandatory metadata of
a test case. DM agreed that's the latter was a nice solution.
SM pointed that there is a general agreement that identifying test
assertions is necessary, but was of the opinion that listing them is
not; e.g., in a formal language description, listing all the test
assertions derived from this description seems a waste of time. PC
precised that in his mind, the listing could be virtual, that is, as
long as there is a way to get such a list (even by using a processor in
the middle), the CP would passed. This made DM point that something
should be said about risk of combinatory number of tests assertions for
formal languages, if they were not well precised.
KD then asked if there were any experiences in the WG about a good list
of test assertions; PC pointed to the SOAP1.2 list (that he will
circulate to the WG list), and SM to the test assertions matrix of the
XML TS (that links test assertions back to the specification). KD then
asked if there were examples of specification where it was not possible
(or very hard) to extract test assertions, and if we could draw
conclusions from these examples; although nobody had such an example, PC
proposed it could be useful to link from TestET to such examples
(possibly in SpecET). More generally, insisting on how benefitial it is
to have Test Assertions identified during the spec development seemed a
good thing to do.
PC will send a write-up of all these test assertions discussion for
further discussion.
-- 
Dominique Hazaël-Massieux - http://www.w3.org/People/Dom/
W3C/ERCIM
mailto:dom@w3.org