W3C home > Mailing lists > Public > www-qa-wg@w3.org > June 2003

Re: TestGL plan needed for Crete !

From: Peter Fawcett <pfawcett@real.com>
Date: Thu, 12 Jun 2003 10:05:01 -0700
Message-Id: <a05111b00bb0e627a3ca2@[172.23.106.41]>
To: Lofton Henderson <lofton@rockynet.com>
Cc: www-qa-wg@w3.org

Nope, just forgot to add the group to the cc line when I did reply to all...

So for the group...


Peter

>Is there any reason that you're not circulating this to QAWG list?
>
>-Lofton.
>
>At 03:34 PM 6/11/03 +0000, you wrote:
>>Howdy,
>>I've got some notes for stuff to talk about for this. Of course I 
>>wont be there :( but at least I can perhaps help spawn some 
>>discussion on some of the current issues as I see it.
>>
>>I reviewed the most recent draft as posted over the last few days 
>>and here's 7 or so topics that are some what general and high level 
>>but still need addressing...
>>
>>So with out further ado:
>>
>>1) Need clearer/better definitions of what Test Framework and Test 
>>Management System mean, what they are composed of, how they differ. 
>>Current definitions are both at the top of section 3: "A test 
>>management systems is a system that organizes and manages test 
>>cases and allows information about test cases to be associated with 
>>the tests. A test framework is a system that assists in the running 
>>of the tests."
>>This may well not be sufficient for non-qa aware folks....
>>
>>2) Conformance requirements for Checkpoint 2.2 is needed. Currently 
>>it's just TBD.  This is especially important as Checkpoints 3.2 and 
>>3.3 as well as others have a dependency on the fulfillment of 2.2 
>>(and is it a good thing (tm) to have dependencies such as this...)
>>
>>3) Large parts of the intro have been re-written to address 
>>comments and issues that had  come up. I'm hoping that is better 
>>but it needs to be reviewed a lot to be sure that it address 
>>concerns that folks had (the majority of the concerns had to do 
>>with clarity, fixing errors from copying parts of the intro of 
>>other Gl docs and not having it fully integrated, arranging the 
>>intro to use the same format/layout as the rest of the GL family 
>>and toning down a few claims that  were a bit over reaching (like 
>>claims of completeness in test suites)). My feeling as author of 
>>the re-write is that it still needs a bit of work... and I'd love 
>>some comments on what others feel still needs work and what it 
>>needs.
>>
>>4) I don't think that Lynne's suggestions about differences between 
>>'interop' testing and test-plans for finished specifications has 
>>been fully addressed. I believe that there is some mention of this 
>>in the spec now but it may need more emphasis (in the intro or some 
>>where else).
>>
>>5) Many of these Guidelines as written are still quite hard to 
>>verify or validate in any way. This brings up the bigger issue of 
>>what can we say that can be "verified" but that doesn't place 
>>restrictions on Test Authors. Not placing restrictions on test case 
>>authors was discussed at the F2F in Boston and every one agreed in 
>>theory that we are not telling people 'how' to write test suites, 
>>just what properties should exists for a quality test suite (focus 
>>on what  should exist when the suite is done rather than focusing 
>>on the process of creation). But even then it's hard to see how 
>>some of these things can be verified, what objective criteria do we
>>use, what do we measure...
>>
>>6) Many (most) of the priorities are new so they need to be checked 
>>to verify that they are correct in every one's opinion.
>>
>>7) Some checkpoints do not have a rationale and some of the 
>>rationale's need more details ( The  only one I see that doesn't 
>>have any at all is Checkpoint 6.2).
>>
>>Have a good time in Crete, I hope it's nice and productive.
>>
>>Thanks,
>>Peter
>>
Received on Thursday, 12 June 2003 13:05:05 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:13 GMT