Re: RDF Core test driven development and QA Test Doc

I'd like to add a comment on this test-led approach as I perceived it in 
the context of RDFcore work.  In many ways I have found it to be valuable, 
and have used similar ideas in other projects I've worked on recently.

One thing I found problematic with the RDFcore test cases was the lack of 
an easily accessible human-readable form -- having the test case data split 
between different files, and file formats that didn't always display fully 
when viewed in a browser.  While I appreciate the value of having 
machine-readable test data, I think the value making it easy also for 
humans to read has been underestimated.

The test data serves two purposes:  (a) to provide some (incomplete*) basis 
that an application satisfies the specification, and (b) to convey to human 
readers in an unambiguous fashion an example of what the working group 
intended to specify.  I feel that purpose (a) has been promoted to the 
detriment of purpose (b).

<aside>
(*)  I say incomplete, because I think it's fallacious to assume that the 
test case suite provides complete coverage of the specification.  We tended 
to create test cases to reflect awkward decisions, not to reflect every 
possible feature of the language.
</aside>

I think it is possible to have one's cake and eat it, though.  With just a 
little additional work, I think the test case data could be prepared in 
such a way that both human- and machine-readable forms can be generated.  I 
think a very simple RDF vocabulary, encoded using Notation 3, could be used 
to describe and discuss test cases from which alternative forms could be 
derived.  We've already seen something similar done for document issue 
tracking.

#g
--

At 16:06 02/01/04 +0100, Jeremy Carroll wrote:


>I am hoping to send some comments to the www-qa list concerning the 
>QAF  later
>today.
>
>I think it may be timely to highlight one comment on the Test Guidelines
>concerning the functional analysis (and overall framework for testing)
>described in that document.
>
>(The document I have reviewed is an editors draft:
>http://www.w3.org/QA/WG/2003/10/TestGL-20031020.html
>)
>
>The comment currently reads:
>
>[[
>Functional Analysis and Test Driven Development
>
>  At times, RDF Core used a test driven specification development 
> methodology.
>
>Issues from the issue list were resolved by agreeing test cases.
>
>The editors then had complete freedom to write text which conformed with the
>test cases. (The text was later reviewed, so the freedom was not as excessive
>as it seems).
>
>Examples can be found both regarding syntax and semantics.
>
>A syntax example is rdfms-empty-property-elements[1] which was resolved with
>these words:
>RESOLUTION: the consolidated test cases represent RDFCore's decision on
>   this; the issue can be closed once those test cases are all in one
>   place.
>
>
>The test cases can be found in this directory [2]. As far as I can tell, this
>predates the first editor's draft of the revised grammar, and modified the
>old grammar. i.e. that this decision, does not follow your methodology at
>all, is a test-focussed decision, and was good.
>
>A semantics example is rdfms-identity-of-statements [3] for which the issue
>resolution is but a single test case:
>   Resolution: The RDFCore WG resolved:
>
>   <stmt1> <rdf:type> <rdf:Statement> .
>   <stmt1> <rdf:subject> <subject> .
>   <stmt1> <rdf:predicate> <predicate> .
>   <stmt1> <rdf:object> <object> .
>   <stmt2> <rdf:type> <rdf:Statement> .
>   <stmt2> <rdf:subject> <subject> .
>   <stmt2> <rdf:predicate> <predicate> .
>   <stmt2> <rdf:object> <object> .
>   <stmt1> <property> <foo> .
>
>       does not entail:
>
>   <stmt2> <property> <foo> .
>
>
>Just perfect! (The syntax used in the test case is a muddle, but all the WG
>members could understand it. The test case will have been sanitised into the
>RDF Test Cases somewhere. The RDF Semantics documents reflect this. Using
>test cases as part, sometimes the only part of, the issue resolution process
>brings clarity and openness. It leaves the editors with large discretion that
>permits the documents to be of higher quality.
>
>In as much as the Test Guidelines and the QAF prohibit and/or obstruct this
>behaviour I suggest that the QAWG has got it wrong, and needs to start over.
>
>[1]
>http://www.w3.org/2000/03/rdf-tracking/#rdfms-empty-property-elements
>[2]
>http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements
>[3]
>http://www.w3.org/2000/03/rdf-tracking/#rdfms-identity-of-statements
>]]
>
>I note that at your telecon on Monday, Sandro will be presenting stuff about
>WebOnt and RDF Core approach to testing. I am available to attend if you
>would like. .
>
>Sandro is likely to be friendlier than me, and I suspect has seen enough of
>how both groups operate to adequately inform you.
>
>Sandro was involved in the latter half of OWL's development and at the very
>end of RDF's. I have been a member of WebOnt throughout its life. My
>involvement with RDF Core postdates the syntax decision above, but I believe
>I voted on the semantics resolution quoted.
>
>Jeremy Carroll

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Received on Sunday, 4 January 2004 05:42:25 UTC