Draft Minutes, 2002-03-14 Telephone Conference

Attached, please find the minutes.

Best,

/Dimitris

---


QA Working Group Teleconference
Thursday, 14-March-2002
--
Scribe: Dimitris Dimitriadis (dd)

Attendees:
(dd) Dimitris Dimitriadis (Ontologicon)
(KD) Karl Dubost (W3C, WG co-chair)
(PF) Peter Fawcett (Real Networks)
(KG) Kirill Gavrylyuk (Microsoft)
(DH) Dominique Hazael-Massieux (W3C - Webmaster)
(LH) Lofton Henderson (CGMO - WG co-chair)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(OT) Olivier Thereaux (W3C - systems)
(CK) Colleen Kelly (Microsoft)

Regrets:
(DD) Daniel Dardailler (W3C - IG co-chair)
(JM) Jack Morrison (Sun)

Absent:
(KH) Katie Haritos-Shea (DOC)
(AT) Andrew Thackrah (Open Group)

Summary of New Action Items:
No new action items

Previous Telcon Minutes:
Not published yet (20020314)

Minutes:


Agenda:
1.) Roll call
2.) Any WG web site/logistical topics
3.) UAAG test suite discussion
LH wrote the agenda just after the mail exchange between Gundersen, 
Jacobs, Dardallier and Henderson. Asked for
topic for discussion with QA WG. Resolution: some of the QA WG join a 
UAAG call. They will not be on today's
telcon. Who might be interested? Interesting from specification 
perspective (intricate performance requirements),
test material provides interesting technical problems.

Potentially interested: Dimitris Dimitriadis, Lofton Henderson, Kirill 
Gavrylyuk, Karl Dubost

4.) Framework document schedule
LH: We aimed at publishing in TR around April 1, progress is a bit 
slower, Editors propose another WG cycle before
publication in the first week of April, target 2-3 weeks after that for 
TR publication.

dd: proposes that commentability is enhanced
LH: some discussion around that. open to suggestion on how to get more 
lively discussion

         - new parts and their progression
LR: Spec guidelines
the draft is a preliminary start for what the document will be. 
describes parts in detail (scribe's phone gave up
here). by identifying information in the spec, you enhance traceability 
of tests as well as automatically
generating tests. Lynne sketched checkpoints and guidelines, dimitris 
concentrated on spec granularity and what that makes
possible.

LH: guildeines 4-9 will break down into more guidelines
LR: anatomy breaks down into more guidelines as well
LH: intra-ed drafs will have fleshed out parts of the new document 
versions

dd: (2b Process&Operations-exTech)
first part gives an account of existing test suites, second part 
monitors checkpoints and guidelines (in 2a) and
investigates how much has been implemented, and gives one example per 
implementation if it exists

         - next publication date

5.) Continued issue#55 processing (per [1])
LH: checkpoints in operational guidelines one by one. the issues list 
won't be rady until this afternoon. issue 55
says look at the checkpoints snd confirm their priorities. We should be 
looking at the 3/11 doc, not the 2/25 one.
We have gotten up to chkpoint 4.2

How do we want to handle the dialogue about chkpoint 1.1 raised by Mark?

MS: summary
first issue: wording on test suites. you always want to make sure that 
you ask for something which is reviewed, we
want to move up the aspect of reviewing the TS in the table.
second point: test assertions, only mentioned in level 6 presently. ms: 
we cannot mention test suites without test
assertions. we mention test suites in level 2 (preliminary) in the more 
complete point (4) we should mention test
assertions. concerned that test assertions are done. the issue is not if 
the assertions are part of the
specification. point: if a test suite, not necessarily complete, exists, 
so should test assertions.

associated issue: we talk about having prio 1 as a commitment to level 3 
in the table, nothing else is said about
the rest of the table, in that case the table should not be published.

dd: stresses that improved specification authoring will expose test 
assertions and failing to meet them.

KG: proposal: agree about test assertions at level 2 (wg should produce 
at least a subset of assertions). levels 5
and 6 should be swapped. do not agree with the comment about review. 
reviews is essentially not done by WG, but
after tests are run.

dd: tests need to be reviewed by the WG, if it's a deliverable.

KG: we don't say that the WG has tests as a deliverable, only to have a 
TS as a deliverable.

dd: WG needs to at least put a "stamp of approval" by the WG

MS: agrees. asking for a deliverable implies reviewing quality (also in 
the TS case)

LR: Wg makes sure that there is a QA review on the TS, whether they do 
it or delegate it to others. when done,
they accept the deliverable (the TS)

KD: if someone accepts to do the review externally, how can we assess 
the quality of the work?

[jack joined the call]

KG: what should the wording be? we need some wording that ensures that 
the TS is reviewed (externally/internally)
and that the TS review process is of good quality

LH: had a reply to the test assertions thread. we need to be careful 
about overly general statements. svg suite
is diagnostic suite, not checking (regarded as good, but does not 
contain list of things to test). we need to be
careful about what we mean with test assertions.

MS: go from spec to test assertion, then to test suite, don't go 
directly to TS

LR: test issues will be adressed in the fourth document

MS: you should be able to trace tests against assertions

LH: there is a wide spctrum of improvements to be made, but it is not 
easy to see how this can be done

MS: do we want to think about the issue about having a table which 
requires level 3, wihtout saying anything about
the rest of the table?

KG: agrees

LH: we should have some convergence and take it off-line for the time 
being (mail)

LH: chkpoint 4.2 is where we were last time.
4.2 listed as a prio 1 chkpoint. previoulsy 4.1 was prio one and was 
changed to prio 2
4.2 - produce a qa document is listed as prio 1. it stays prio 1 (no 
objections)

4.3 - in the qa process doc, define the means of qa related 
communication. is this a necessary part of satsfying
4.2?

KG: no

LH: issue 56, you have to have objective test critera, what is the 
objective criterion for 4.2? in the 2b doc, you
could give examples, if they exist, but what if they do not? what 
interpretation of the 2a chkpoint do you give?

LH: are 4.3, 4.4, 4.5 lower prios?

4.3:

MS: propose that having a qa moderator is necessary

KG: should qa moderator be prio 1 and qa task force be prio 2?

MS: yes

agreed

LH: 4.4

KG: reason for prio 2 is that the framework (how to develop docs and use 
the tests) may not be needed.
perhaps wrong, should it be prio 1?

ALL: yes

KG: 4.5 specifying policy for branding materials. prio 3. keeps 3 but 
strike out "if applicable"

RESOLUTION: keep prio 3 and remove "if applicable"

gdline 5, ensure that ts is documented and reusable.

LR: remove the "accredited third party" phrasing

MS: you don't necessarily need third party certification, if someone 
wants third party crediation, what's the prob?

LR: get rid of accredited 3rd party

KG: agree, keep reusable

LR: then I see it as a prio 1

KG: ok, let's raise it to prio 1

LH: extra telcon: three absents, lh, dd, jc if we go for next thursday. 
when should we have the extra telcon?

agreed to have the extra telcon normal time next thursday

6.) Adjourn

adjourned 3 minutes past scheduled time

Received on Thursday, 14 March 2002 16:47:58 UTC