- From: Dimitris Dimitriadis <dimitris@ontologicon.com>
- Date: Wed, 14 Jan 2004 19:46:40 +0200
- To: www-qa-wg@w3.org
All, Below please find today's draft minutes for your comments. Best, /Dimitris --- QA Working Group Teleconference Monday, 14-January-2004 -- Scribe: Dimitris Dimitriadis Attendees: (PC) Patrick Curran (Sun Microsystems) (dd) Dimitris Dimitriadis (Ontologicon) (KD) Karl Dubost (W3C, WG co-chair) (DH) Dominique Hazaël-Massieux (W3C) (LH) Lofton Henderson (CGMO - WG co-chair) (LR) Lynne Rosenthal (NIST - IG co-chair) (MS) Mark Skall (NIST) (AT) Andrew Thackrah (Open Group) Guests (DM) Dave Marston, IBM Regrets: (MC) Martin Chamberlain (Microsoft) (SM) Sandra Martinez (NIST) (VV) Vanitha Venkatraman (Sun Microsystems) Absent: Summary of New Action Items: [Format: AI-YYYYMMDD-N Who What DEADLINE! ] AI-20040114-1 PC/dd Generate issues list for TestGL from email 20040119 AI-20040114-2 PC/dd Generate draft for TestGL 20040126 Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0052.html Previous Telcon Minutes: http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0041.html (draft) Minutes: 1.) roll call 11am EDT, membership 2.) routine business - note that 1/19 is cancelled - Future telecons [0] (note 1/26) - TP2004 deadlines reminder [1] DM: contemplating sending in supplement to proposal for WWW2004 conference. 3.) TestGL LH: no new text or organized issues list to look at. Propose two topics: (1) look at the project itself, (2) Jeremy Carrol (JC) thread, it raises issues about TestGL - (1) project future, plan, and schedule LH:we're already 8 months after our last publication, W3C wants to see 3 nominally PC: problem - lack of organized issues list, we now have to read minutes and work from that. PC and dd to comb through email and come up with an issues list. LH: if we can make it look at a document and flag issues, we will prevent misunderstandings PC: had not focussed on the boilerplate, if this adds value can do so. LH: words for introductory section lacking as well PC: mail with dd over the next week LH: patrick, you mentioned it is not plausible that vanitha contribute to the issues list? PC: may ease up in the next few weeks, not before that. - (2) Carroll thread & test-driven dev't [2] LH: we can extract a few specific issues. PC: high level points - keeps mentioning test driven development process. seems to be saying this is what he did (started with a test, used the process of defining a test to attack issues). seems to be saying the way TestGL is written (especially analyze specifications, generate tests etc.) - requires a model of development and would have prohibited them to work. problem is that he does not really talk of conformance testing. if what they have come up with is conformance testing, it's to be viewed as a side effect. I think we should stick to our version - conformance testing. DH: I think the point is that the testing material is not conformance testing as it is. do we really want to restrict TestGL to conformance testing? LH: in the Crete minutes I think the idea was that TestGL should be restricted in scope KD: different points of view. in a sense JC is saying that he needs a test suite to develop the specification. he seems to think qa wg is in the end of the process (conformance testing), and wants to pay respect to the start of it (writing a specification) LH: low level comment: looking at SpecGL we, in Crete and Boulder, allowed for using test materials in specification development. we got away from standardizing process to standardizing results. coverage maps are of no interest to someone who develops specifications using test development. for conformance testing we need a concept of coverage mapping. if we try to broaden TestGL, it could be really tricky. the broader point is I went through the introduction searching for "test materials" and "test suite", half of the occurences are about conformance. we seem to have had it in mind all along. PC: I think it's clear that our primary focus is conformance testing, interoperability would perhaps be next. DM: we already have materials indicating the that is indeed the primary focus. PC: another theme in what JC is saying is that the language we use appears to be overly restrictive. what he says is that language formulation would exclude many groups from passing having used test materials. DD: free up language to allow for many kinds of uses. DH: one interesting thing in JC's reaction is that he's alocacted lots of time to comment, we should therefore investigate thouroughly. LH: think that one solution is to clarify our scope, writing guidelines for a somewhat different scope. DH: I'm saying we should think about broadening our scope in order to accomodate for more kinds of use. LH: agree in principle, as a practical matter we should limit things to our resources. in this generation of specs, we're not going to try to place conformance requirements. DD: we should still aim for our original intention, which is conformance, but ease up language to allow for more uses. MS: at the broader level, little bit uncomfortable with changing the scope, we're already behind in terms of writing guidelines for conformance testing. we should stick to our guns. PC: conformance testing is difficult, expensive, not really interesting, but that is what it is; if we belive there is value in it, we should aim for it. KD: Patrick, in the way testing guidelines are written now, is it possible to define profiles that will make the guidelines usable for conformance test suite, for test driven development, and other uses? for example, use these guidelines for this kind process (test driven specification development), these for that kind of process (improve conformance/test interoperability). PC: seems to be rather difficult. MS: one more thing about conformance testing is perhaps a misnoma, it implies restrictions. in general, the outcome is improve quality in implementations, which is qa. it is a less well known impact of doing strict conformance testing. PC: it's dual purpose - it tests conformance but also improves quality. LH: if not careful defining scope, we may end up with something useless to the WG. Looking at the three guidelines docs, almost none is related to test suites, but related to making specs clear and testable. DH: conformance testing is really useful, but we're developing test guidelines, so we have to somewhat limit the scope if we want to finish at some point. do we do conformance testing or more "loose" testing? in the W3C it's interoperability, not conformance. DD: shouldn't it be the other way around? LH: dom, what do you propose as the way forward? go back to scratch, or continue what we're doing while acknowledging that we're doing only a subset of what we ideally want to be doing in the future? DH: look at the checkpoints and look at what ones are only conformance testing, and what ones are border cases. I'd need to see a new draft of TestGL before knowing how much time it would take. PC: no conclusion on the JC issues, pending on the next version of the draft and issues list. 4.) TA list for SpecGL (resuming at GL8) - TA list for 12-sept WD [3] - issues: [4a], [4b] - previous results: [5a], [5b], [5c] - DM SpecGL issue on CP7.4 [6] 5.) Adjourn meeting adjourned 4 minutes past the hour [0] http://www.w3.org/QA/Agenda/#agenda-telcon [1] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0064.html [2] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0000.html [3] http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/att-0018/ Assertions.html [4a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0032.html [4b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0034.html [5a] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0015.html [5b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Dec/0065.html [5c] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0041.html [6] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0043.html
Received on Wednesday, 14 January 2004 12:46:57 UTC