Re: Further Feedback on SPARQL 1.1 Test Cases

Hi Rob,

This is in response to http://lists.w3.org/Archives/Public/public-rdf-dawg-comments/2011Feb/0024.html

First of all, apologies for taking so long to reply and thanks for your thorough and helpful remarks.
We are getting back to the test suite at the moment and were looking into your comments.
Please note that the test suite is work under progress and that particularly tests with approval status 
"dawgt:NotClassified" are not yet approved by the group. Nevertheless your detailed feedback (also on those test cases) is appreciated!

 > Bugs/Issues:
 > 
 > 1 - In the delete/ tests delete-pre-03.ttl uses example.org as the email
 > address domain while in the corresponding delete-post-03f.ttl file it
 > changes to example.com
 > Additionally the prefix mapping for the default prefix : is defined
 > differently (http://example.org in the delete-pre-03.ttl and http://example
 > in the delete-post-03f.ttl version)
 > None of the test cases in the delete/ directory are capable of making this
 > change to the data so I assume the expected result default prefix mapping
 > and data should be fixed to use example.org

These should be fixed now.

 > 2 - Some tests have malformed SPARQL Results XML as their expected results.
 >  Some try to bind variables in the <results> element which aren't declared
 > in the <head> element and some are missing the <head> element completely. 
 > Affected Results files include the following:
 >
 > day01.srx
 > minutes01.srx
 > month01.srx
 > now01.srx
 > rand01.srx
 > seconds01.srx
 > substring01.srx
 > timezone01.srx
 > tz01.srx
 > year01.srx

This should be fixed now.

 
 > 3 - Some manifests mention non-existent tests:
 >
 > functions/notin01.rq

This should be fixed now.

 > negation/temporalProximity02.rq

The respective test case temporal-proximity-by-exclusion-minus-1
is commented out in the manifest, cf.
 http://www.w3.org/2009/sparql/docs/tests/data-sparql11/negation/manifest.ttl

 >
 > Comments:
 >
 > Some of the tests for MINUS and NOT EXISTS are very complex and from an
 > implementer point of view it is very difficult to determine where my
 > implementation is going wrong on these cases since their complexity makes
 > them hard to debug.
 > 
 > I'm not suggesting that the test cases should be removed but some
 > intermediate test cases between the very simple and very complex test cases
 > would be nice and would aid understanding and debugging.

We are currently looking into this, in the course of checking coverage of the test suite, 
but would be grateful for concrete proposals for test cases, if you find something missing.

As for your further comments below on the test case syntax, this should also be fixed now.

Please respond indicating whether you feel this response has adequately answered your comment.

Thanks and best regards,
Axel, on behalf of the WG

 > -------- Original Message --------
 > Subject: Re: Inconsistent syntax in testcase manifest
 > Date: Wed, 23 Feb 2011 11:14:12 +0000
 > From: Rob Vesse <rav08r@ecs.soton.ac.uk>
 > To: Birte Glimm <birte.glimm@comlab.ox.ac.uk>
 >
 > Hi DAWG
 >
 > I notice that in some of the per-directory manifests graphs in the datasets
 > for Update Evaluation Tests are identified like so:
 >
 > ut:graphData  [ ut:data <spo.ttl> ;
 >                                rdfs:label "http://example.org/g1" ] 
 > 
 > But in others like so:
 >
 > ut:graphData [ ut:graph <delete-pre-01.ttl> ;
 >                                rdfs:label "http://example.org/g1" ]
 >
 > So far I have found this in the manifest for delete-data but there may be
 > others I haven't got as far as running yet
 >
 > The first form seems to be most commonly used (and is consistent with Query
 > Evaluation Tests using ut:data to identify data files) so I think the 2nd
 > form should be changed to match the first.
 >
 > I assume that this is supposed to mean the same thing but it would be nice 
 > if the manifests were consistent as it makes writing a test harness just
 > that little bit more frustrating
 >
 > Thanks,
 >
 > Rob Vesse

Received on Sunday, 7 August 2011 21:06:00 UTC