[Fwd: Re: RDF Core test driven development and QA Test Doc]

This is a msg originally from Graham Klyne in a thread that I started.
I neglected to include the QAIG in the cc line.

Note: other messages in this thread can be found in the QAWG list, 
including an initial response from Lofton.

A further comment is that when we started on the OWL Test Cases the desire 
for an easily human readable form of the tests motivated including the 
tests in-line in the HTML. (This is essentially Graham's last paragraph below).

The OWL Test work took the RDF Test work as its prototype; however the 
WebOnt WG did not use RDF Core's test driven spec development methodology.

Jeremy


-------- Original Message --------
Subject: Re: RDF Core test driven development and QA Test Doc
To: w3c-rdfcore-wg@w3.org


I'd like to add a comment on this test-led approach as I perceived it in
the context of RDFcore work.  In many ways I have found it to be valuable,
and have used similar ideas in other projects I've worked on recently.

One thing I found problematic with the RDFcore test cases was the lack of
an easily accessible human-readable form -- having the test case data split
between different files, and file formats that didn't always display fully
when viewed in a browser.  While I appreciate the value of having
machine-readable test data, I think the value making it easy also for
humans to read has been underestimated.

The test data serves two purposes:  (a) to provide some (incomplete*) basis
that an application satisfies the specification, and (b) to convey to human
readers in an unambiguous fashion an example of what the working group
intended to specify.  I feel that purpose (a) has been promoted to the
detriment of purpose (b).

<aside>
(*)  I say incomplete, because I think it's fallacious to assume that the
test case suite provides complete coverage of the specification.  We tended
to create test cases to reflect awkward decisions, not to reflect every
possible feature of the language.
</aside>

I think it is possible to have one's cake and eat it, though.  With just a
little additional work, I think the test case data could be prepared in
such a way that both human- and machine-readable forms can be generated.  I
think a very simple RDF vocabulary, encoded using Notation 3, could be used
to describe and discuss test cases from which alternative forms could be
derived.  We've already seen something similar done for document issue
tracking.

#g
--

At 16:06 02/01/04 +0100, Jeremy Carroll wrote:


 >I am hoping to send some comments to the www-qa list concerning the
 >QAF  later
 >today.
 >
 >I think it may be timely to highlight one comment on the Test Guidelines
 >concerning the functional analysis (and overall framework for testing)
 >described in that document.
 >
 >(The document I have reviewed is an editors draft:
 >http://www.w3.org/QA/WG/2003/10/TestGL-20031020.html
 >)
 >
 >The comment currently reads:
 >
 >[[
 >Functional Analysis and Test Driven Development
 >
 >  At times, RDF Core used a test driven specification development
 > methodology.
 >
 >Issues from the issue list were resolved by agreeing test cases.
 >
 >The editors then had complete freedom to write text which conformed with the
 >test cases. (The text was later reviewed, so the freedom was not as excessive
 >as it seems).
 >
 >Examples can be found both regarding syntax and semantics.
 >
 >A syntax example is rdfms-empty-property-elements[1] which was resolved with
 >these words:
 >RESOLUTION: the consolidated test cases represent RDFCore's decision on
 >   this; the issue can be closed once those test cases are all in one
 >   place.
 >
 >
 >The test cases can be found in this directory [2]. As far as I can tell, this
 >predates the first editor's draft of the revised grammar, and modified the
 >old grammar. i.e. that this decision, does not follow your methodology at
 >all, is a test-focussed decision, and was good.
 >
 >A semantics example is rdfms-identity-of-statements [3] for which the issue
 >resolution is but a single test case:
 >   Resolution: The RDFCore WG resolved:
 >
 >   <stmt1> <rdf:type> <rdf:Statement> .
 >   <stmt1> <rdf:subject> <subject> .
 >   <stmt1> <rdf:predicate> <predicate> .
 >   <stmt1> <rdf:object> <object> .
 >   <stmt2> <rdf:type> <rdf:Statement> .
 >   <stmt2> <rdf:subject> <subject> .
 >   <stmt2> <rdf:predicate> <predicate> .
 >   <stmt2> <rdf:object> <object> .
 >   <stmt1> <property> <foo> .
 >
 >       does not entail:
 >
 >   <stmt2> <property> <foo> .
 >
 >
 >Just perfect! (The syntax used in the test case is a muddle, but all the WG
 >members could understand it. The test case will have been sanitised into the
 >RDF Test Cases somewhere. The RDF Semantics documents reflect this. Using
 >test cases as part, sometimes the only part of, the issue resolution process
 >brings clarity and openness. It leaves the editors with large discretion that
 >permits the documents to be of higher quality.
 >
 >In as much as the Test Guidelines and the QAF prohibit and/or obstruct this
 >behaviour I suggest that the QAWG has got it wrong, and needs to start over.
 >
 >[1]
 >http://www.w3.org/2000/03/rdf-tracking/#rdfms-empty-property-elements
 >[2]
 >http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements
 >[3]
 >http://www.w3.org/2000/03/rdf-tracking/#rdfms-identity-of-statements
 >]]
 >

Received on Sunday, 4 January 2004 18:10:03 UTC