Re: RDF Core test driven development and QA Test Doc

Hi Brian

concerning:

Jeremy:
> >
> > In as much as the Test Guidelines and the QAF prohibit
> >  and/or obstruct this
> > behaviour

Brian:
> Raised eyebrows!  This document is restrictive?  It is saying that a WG 
> is not allowed to do something, rather than advising on good practice?

From 

http://www.w3.org/TR/2003/WD-qaframe-test-20030516/#scope
[[
The scope of this specification is a set of requirements for Test Materials 
(TM) that, if satisfied, will enhance the usability and clarity of the test 
materials.
]] 

From
http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/guidelines-chapter#Ck-TM-plans-in-charter
[[
for any Test Materials that it plans to produce or adopt, the WG MUST define 
its commitment level to QA Framework: Test Guidelines -- A, AA, or AAA.
]]

Moreover
http://www.w3.org/TR/2003/WD-qaframe-test-20030516/#gl1
[[
In order to determine the testing strategy or strategies to be used, a 
high-level analysis of the structure of the specification (the subject of the 
test suite) must be performed. 
]]
(I believe this is a non-normative must)

http://www.w3.org/TR/2003/WD-qaframe-test-20030516/#gl1cp3
[[
Checkpoint 1.3. Analyze the structure of the specification, partition it as 
appropriate, and determine and document the testing approach to be used for 
the test suite as a whole and for each partition. [Priority 1]
]]
(I believe this to be a normative instruction)

Both of these are applicable to A AA and AAA conforming test suites and cannot 
be done if the specification is constructed to follow the test suite.

Of course, one way to approach this is to commit to AAA conformance to the 
test guidelines and then fail to deliver on that commitment.
cf
http://www.w3.org/QA/WG/charter.html#Scope

An alternative approach would be to use one test suite constructed ala RDF 
Core, then build the spec, and then build a second test suite ala QAWG.

I have been told I have misunderstood but none of the crucial text has 
changed:
http://lists.w3.org/Archives/Public/www-qa/2003Jul/0020
see particularly (a) concerning "waterfall model"
some of the text that Karl quoted such as

  "This makes sense, since it is natural for test suites
        and implementations to develop in parallel - each is
        a help to the development of the other. "

seems to miss the point about test led spec development - the tests can be the 
very first thing, leading the implementations - that is how RDF Core worked.

--- drift ---

Looking at that thread in July again I see that:
a) I made a comment concerning Ops and Spec guidelines too
b) that comment was not formally addressed before Ops and Spec guidelines was 
advanced to CR
c) is one that I would have formally objected about if given the chance
d) that the other comments I made (concerning test guidelines) have not yet 
been formally addressed
e) that the QAWG issues list at the time of the request for advance to CR did 
not list any open issues, hence suppressing the fact that there were still 
open comments on the test guidelines.
f) while the comment was after the LC period for both the Ops and Spec 
guidelines, it was timely, given that it refers to a recent decision
g) the process doc is clear that late comments must be formally addressed

I should formally object to this, will do so on the appropriate list when I 
have time.

http://lists.w3.org/Archives/Public/www-qa/2003Jul/0023
[[[
> 
>>
>> Checkpoint 4.1 is wrong and should be deleted. This is entirely out of 
>> scope for a W3C WG.
> 

cf:
http://lists.w3.org/Archives/Member/w3c-i18n-wg/2003Apr/0002.html
(member only link)
On issue C035, Recommendations and W3C Policy
[[
   RESOLVED: Rec track documents can ...
]]
(Sorry since this is a public list I am not sure what I may or may not copy 
from that member only message, you will need to go and look).
I believe it impacts all your documents.
I would be interested in continuing this part of the discussion on a member 
only list e.g. w3c-archive
]]]

The need to formally object to my comments being ignored is particularly clear 
straight after I have repeated that very same comment, fearing that I had 
never formally made it. Life would be somewhat easier if the QA WG's comments 
list and the  QAIG list were different.

see
http://lists.w3.org/Archives/Public/www-archive/2004Jan/att-0000/#x4.4


Jeremy

Received on Saturday, 3 January 2004 16:40:21 UTC