Re: RDF Core test driven development and QA Test Doc

Lofton Henderson wrote:

> I know Karl has proposed (previous message in this thread) that the 
> test-driven development of OWL specifications means, by definition, that 
> you pass the checkpoint we're talking about.
> 
> Without addressing at that view, I'd like to look at this from a 
> different angle, starting with a clarification.  Again, these are my own 
> views, as the QAWG has not yet taken up the question(s) you raise.  (But 
> will do so in a near-future telecon.)
> 
> At 09:07 PM 1/6/04 +0000, Jeremy Carroll wrote:
> 
>> [...]
>> Test GL:
>> >> [[
>> >> Checkpoint 1.3. Analyze the structure of the specification, partition
>> >> it as appropriate, and determine and document the testing approach to
>> >> be used for the test suite as a whole and for each partition.
>> >> [Priority 1]
>> >> ]]
>>
>> This checkpoint defines method - the testing approach is determined as 
>> a result of analysing the specification.
>> That might not be what the QAWG intends but that is what the current 
>> WD and editors draft both say.
> 
> 
> We have been trying to move away from defining the method or process, 
> which was prevalent in the earliest drafts.  Instead, we are trying to 
> state the requirements in terms of testable characteristics of the test 
> suite (TS).
> 
> Note that the normative content of CP1.3 is not in the statement quoted 
> above, but rather it is contained in the Conformance Requirements section: 
> 
>     "The scope, goal, and purpose of the test suite as a whole, and
>     where appropriate of each logical 'partition' of the test suite, and
>     the mapping between such partitions and sections of the
> specification, must be identified and documented"
> 
> [Note.  This WG-only draft is raw, and "must" should be read as "MUST" 
> -- i.e., that's a normative must above.]  


That's much better - I still disagree with it :), but that seems more a 
matter of opinion.

I would not object to such text. (I might have misunderstood the editor's 
draft division between normative and informative)

> 
> So we've improved the ConfReqs -- the only normative part of the 
> checkpoint -- so they can now be read as testable conditions on a 
> conforming conformance test suite.  [We should continue to align the 
> CPs' statements, so they don't misleadingly suggest "process", and we 
> should fiddle with the wording of the Rationale, for same reason.]
> 
> You then said...
> 
>> (So while the checkpoint above could be written more declaratively e.g.
>> "There should be a systematic relationship between the organization of 
>> the test suite and the sections of the specification", this would 
>> still have been a problem for RDF Core, in my view. Such a change, 
>> while an improvement, does not go far enough to get my support).
> 
> 
> This is getting close to the ConfReqs for CP1.3, which are the normative 
> parts of the checkpoint.  Except notice that CP1.3 ConfReqs only 
> requires that the relationship be documented, whatever it is, whereas 
> your proposed rewording seems to require a systematic relationship.  
> (Since you said, "still a problem", I'm assuming that you're thinking 
> that we intend to require that the structures be parallel, isomorphic, 
> or something like that.  IMO, CP1.3 does not require that, but 
> encourages its consideration by requiring documentation.)
> 
> IMO, for the OWL TS to pass CP1.3 would be trivial, if indeed it doesn't 
> already pass (as Karl claims) -- it would be a small addition (not even 
> necessarily Normative), to the OWL TS document.  Maybe it's even already 
> there?  (I read the document, but don't remember all the details). 
> 


I agree it would have been low cost for both SW WGs to have conformed with 
that text, as long as 'partition' is understood as high-level partition 
rather than low-level. (I think that's the intent - later text seems to go 
for the detailed relationship between tests and test assertions).

In fact "each *high-level* logical 'partition'" might be appropriate text, 
with informative examples of high-level.

I look forward to the next WD of the test work.


Jeremy

Received on Wednesday, 7 January 2004 10:10:18 UTC