Minutes telcon 20040114 (final)

Dear all,

having received no comments, below you will find the final minutes:

Please note that an unrecorded AI (thanks for the catch, Lofton) is for  
myself and Patrick to come up with a schedule for future TestGL work,  
which we will send to the list shortly.

Best,

Dimitris

---

QA Working Group Teleconference
Monday, 14-January-2004
--
Scribe: Dimitris Dimitriadis

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(dd) Dimitris Dimitriadis (Ontologicon)
(KD) Karl Dubost (W3C, WG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO - WG co-chair)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(AT) Andrew Thackrah (Open Group)

Guests
(DM) Dave Marston, IBM

Regrets:
(MC) Martin Chamberlain (Microsoft)
(SM) Sandra Martinez (NIST)
(VV) Vanitha Venkatraman (Sun Microsystems)

Absent:

Summary of New Action Items:
[Format:  AI-YYYYMMDD-N   Who    What    DEADLINE! ]
AI-20040114-1 PC/dd Generate issues list for TestGL from email 20040119
AI-20040114-2 PC/dd Generate draft for TestGL 20040126

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0052.html


Previous Telcon Minutes:  
http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0041.html (draft)

Minutes:

1.) roll call 11am EDT, membership

2.) routine business
        - note that 1/19 is cancelled
        - Future telecons [0] (note 1/26)
        - TP2004 deadlines reminder [1]

DM: contemplating sending in supplement to proposal for WWW2004  
conference.

3.) TestGL

LH: no new text or organized issues list to look at. Propose two  
topics: (1) look at the project itself, (2) Jeremy Carrol (JC) thread,  
it raises issues about TestGL

        - (1) project future, plan, and schedule
LH:we're already 8 months after our last publication, W3C wants to see  
3 nominally
PC: problem - lack of organized issues list, we now have to read  
minutes and work from that. PC and dd to comb through email and come up  
with an issues list.
LH: if we can make it look at a document and flag issues, we will  
prevent misunderstandings
PC: had not focussed on the boilerplate, if this adds value can do so.
LH: words for introductory section lacking as well
PC: mail with dd over the next week
LH: patrick, you mentioned it is not plausible that vanitha contribute  
to the issues list?
PC: may ease up in the next few weeks, not before that.

        - (2) Carroll thread & test-driven dev't [2]
LH: we can extract a few specific issues.
PC: high level points - keeps mentioning test driven development  
process. seems to be saying this is what he did (started with a test,  
used the process of defining a test to attack issues). seems to be  
saying the way TestGL is written (especially analyze specifications,  
generate tests etc.) - requires a  model of development and would have  
prohibited them to work. problem is that he does not really talk of  
conformance testing. if what they have come up with is conformance  
testing, it's to be viewed as a side effect. I think we should stick to  
our version - conformance testing.
DH: I think the point is that the testing material is not conformance  
testing as it is. do we really want to restrict TestGL to conformance  
testing?
LH: in the Crete minutes I think the idea was that TestGL should be  
restricted in scope
KD: different points of view. in a sense JC is saying that he needs a  
test suite to develop the specification. he seems to think qa wg is in  
the end of the process (conformance testing), and wants to pay respect  
to the start of it (writing a specification)
LH: low level comment: looking at SpecGL we, in Crete and Boulder,  
allowed for using test materials in specification development. we got  
away from standardizing process to standardizing results. coverage maps  
are of no interest to someone who develops specifications using test  
development. for conformance testing we need a concept of coverage  
mapping. if we try to broaden TestGL, it could be really tricky. the  
broader point is I went through the introduction searching for "test  
materials" and "test suite", half of the occurences are about  
conformance. we seem to have had it in mind all along.
PC: I think it's clear that our primary focus is conformance testing,  
interoperability would perhaps be next.
DM: we already have materials indicating the that is indeed the primary  
focus.
PC: another theme in what JC is saying is that the language we use  
appears to be overly restrictive. what he says is that language  
formulation would exclude many groups from passing having used test  
materials.
DD: free up language to allow for many kinds of uses.
DH: one interesting thing in JC's reaction is that he's alocacted lots  
of time to comment, we should therefore investigate thouroughly.
LH: think that one solution is to clarify our scope, writing guidelines  
for a somewhat different scope.
DH: I'm saying we should think about broadening our scope in order to  
accomodate for more kinds of use.
LH: agree in principle, as a practical matter we should limit things to  
our resources. in this generation of specs, we're not going to try to  
place conformance requirements.
DD: we should still aim for our original intention, which is  
conformance, but ease up language to allow for more uses.
MS: at the broader level, little bit uncomfortable with changing the  
scope, we're already behind in terms of writing guidelines for  
conformance testing. we should stick to our guns.
PC: conformance testing is difficult, expensive, not really  
interesting, but that is what it is; if we belive there is value in it,  
we should aim for it.
KD: Patrick, in the way testing guidelines are written now, is it  
possible to define profiles that will make the guidelines usable for  
conformance test suite, for test driven development, and other uses?  
for example, use these guidelines for this kind process (test driven  
specification development), these for that kind of process (improve  
conformance/test interoperability).
PC: seems to be rather difficult.
MS: one more thing about conformance testing is perhaps a misnoma, it  
implies restrictions. in general, the outcome is improve quality in  
implementations, which is qa. it is a less well known impact of doing  
strict conformance testing.
PC: it's dual purpose - it tests conformance but also improves quality.
LH: if not careful defining scope, we may end up with something useless  
to the WG. Looking at the three guidelines docs, almost none is related  
to test suites, but related to making specs clear and testable.
DH: conformance testing is really useful, but we're developing test  
guidelines, so we have to somewhat limit the scope if we want to finish  
at some point. do we do conformance testing or more "loose" testing? in  
the W3C it's interoperability, not conformance.
DD: shouldn't it be the other way around?
LH: dom, what do you propose as the way forward? go back to scratch, or  
continue what we're doing while acknowledging that we're doing only a  
subset of what we ideally want to be doing in the future?
DH: look at the checkpoints and look at what ones are only conformance  
testing, and what ones are border cases. I'd need to see a new draft of  
TestGL before knowing how much time it would take.
PC: no conclusion on the JC issues, pending on the next version of the  
draft and issues list.

4.) TA list for SpecGL (resuming at GL8)
        - TA list for 12-sept WD [3]
        - issues:  [4a], [4b]
        - previous results:  [5a], [5b], [5c]
        - DM SpecGL issue on CP7.4 [6]
        
5.) Adjourn
meeting adjourned 4 minutes past the hour

[0] http://www.w3.org/QA/Agenda/#agenda-telcon
[1] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0064.html
[2] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0000.html
[3]  
http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/att-0018/ 
Assertions.html
[4a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0032.html
[4b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0034.html
[5a] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0015.html
[5b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0065.html
[5c] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0041.html
[6] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0043.html

Received on Friday, 23 January 2004 13:42:37 UTC