RDF Core test driven development and QA Test Doc

I am hoping to send some comments to the www-qa list concerning the QAF  later 
today.

I think it may be timely to highlight one comment on the Test Guidelines 
concerning the functional analysis (and overall framework for testing) 
described in that document.

(The document I have reviewed is an editors draft:
http://www.w3.org/QA/WG/2003/10/TestGL-20031020.html
)

The comment currently reads:

[[
Functional Analysis and Test Driven Development

 At times, RDF Core used a test driven specification development methodology. 

Issues from the issue list were resolved by agreeing test cases.

The editors then had complete freedom to write text which conformed with the 
test cases. (The text was later reviewed, so the freedom was not as excessive 
as it seems). 

Examples can be found both regarding syntax and semantics. 

A syntax example is rdfms-empty-property-elements[1] which was resolved with 
these words: 
RESOLUTION: the consolidated test cases represent RDFCore's decision on
  this; the issue can be closed once those test cases are all in one
  place.
    

The test cases can be found in this directory [2]. As far as I can tell, this 
predates the first editor's draft of the revised grammar, and modified the 
old grammar. i.e. that this decision, does not follow your methodology at 
all, is a test-focussed decision, and was good. 

A semantics example is rdfms-identity-of-statements [3] for which the issue 
resolution is but a single test case: 
  Resolution: The RDFCore WG resolved:

  <stmt1> <rdf:type> <rdf:Statement> .
  <stmt1> <rdf:subject> <subject> .
  <stmt1> <rdf:predicate> <predicate> .
  <stmt1> <rdf:object> <object> .
  <stmt2> <rdf:type> <rdf:Statement> .
  <stmt2> <rdf:subject> <subject> .
  <stmt2> <rdf:predicate> <predicate> .
  <stmt2> <rdf:object> <object> .
  <stmt1> <property> <foo> .

      does not entail:

  <stmt2> <property> <foo> .
  

Just perfect! (The syntax used in the test case is a muddle, but all the WG 
members could understand it. The test case will have been sanitised into the 
RDF Test Cases somewhere. The RDF Semantics documents reflect this. Using 
test cases as part, sometimes the only part of, the issue resolution process 
brings clarity and openness. It leaves the editors with large discretion that 
permits the documents to be of higher quality. 

In as much as the Test Guidelines and the QAF prohibit and/or obstruct this 
behaviour I suggest that the QAWG has got it wrong, and needs to start over. 

[1] 
http://www.w3.org/2000/03/rdf-tracking/#rdfms-empty-property-elements
[2]
http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements
[3]
http://www.w3.org/2000/03/rdf-tracking/#rdfms-identity-of-statements
]]

I note that at your telecon on Monday, Sandro will be presenting stuff about 
WebOnt and RDF Core approach to testing. I am available to attend if you 
would like. .

Sandro is likely to be friendlier than me, and I suspect has seen enough of 
how both groups operate to adequately inform you.

Sandro was involved in the latter half of OWL's development and at the very 
end of RDF's. I have been a member of WebOnt throughout its life. My 
involvement with RDF Core postdates the syntax decision above, but I believe 
I voted on the semantics resolution quoted.

Jeremy Carroll

Received on Friday, 2 January 2004 10:07:31 UTC