Re: Further Feedback on SPARQL 1.1 Test Cases

Hi all

In running the complete suite of test cases as it stands currently I have
found some additional bugs/errors in the test cases.  I also have one
general comment about some of the negation test cases.

Bugs/Issues:

1 - In the delete/ tests delete-pre-03.ttl uses example.org as the email
address domain while in the corresponding delete-post-03f.ttl file it
changes to example.com
Additionally the prefix mapping for the default prefix : is defined
differently (http://example.org in the delete-pre-03.ttl and http://example
in the delete-post-03f.ttl version)
None of the test cases in the delete/ directory are capable of making this
change to the data so I assume the expected result default prefix mapping
and data should be fixed to use example.org

2 - Some tests have malformed SPARQL Results XML as their expected results.
 Some try to bind variables in the <results> element which aren't declared
in the <head> element and some are missing the <head> element completely. 
Affected Results files include the following:

day01.srx
minutes01.srx
month01.srx
now01.srx
rand01.srx
seconds01.srx
substring01.srx
timezone01.srx
tz01.srx
year01.srx

3 - Some manifests mention non-existent tests:

functions/notin01.rq
negation/temporalProximity02.rq

Comments:

Some of the tests for MINUS and NOT EXISTS are very complex and from an
implementer point of view it is very difficult to determine where my
implementation is going wrong on these cases since their complexity makes
them hard to debug.

I'm not suggesting that the test cases should be removed but some
intermediate test cases between the very simple and very complex test cases
would be nice and would aid understanding and debugging.

Regards,

Rob Vesse

-------- Original Message --------
Subject: Re: Inconsistent syntax in testcase manifest
Date: Wed, 23 Feb 2011 11:14:12 +0000
From: Rob Vesse <rav08r@ecs.soton.ac.uk>
To: Birte Glimm <birte.glimm@comlab.ox.ac.uk>

Hi DAWG

I notice that in some of the per-directory manifests graphs in the datasets
for Update Evaluation Tests are identified like so:

ut:graphData  [ ut:data <spo.ttl> ;
                                rdfs:label "http://example.org/g1" ] 

But in others like so:

ut:graphData [ ut:graph <delete-pre-01.ttl> ;
                               rdfs:label "http://example.org/g1" ]

So far I have found this in the manifest for delete-data but there may be
others I haven't got as far as running yet

The first form seems to be most commonly used (and is consistent with Query
Evaluation Tests using ut:data to identify data files) so I think the 2nd
form should be changed to match the first.

I assume that this is supposed to mean the same thing but it would be nice
if the manifests were consistent as it makes writing a test harness just
that little bit more frustrating

Thanks,

Rob Vesse

On Sat, 19 Feb 2011 00:20:12 +0000, Birte Glimm
<birte.glimm@comlab.ox.ac.uk> wrote:
> On 17 February 2011 20:23, Jeen Broekstra <jeen.broekstra@gmail.com>
wrote:
>>
>> Hi DAWG,
>>
>> I'm trying to run the SPARQL 1.1 testcases at
>> http://www.w3.org/2009/sparql/docs/tests/ .
>>
>> Unfortunately I run into trouble: at least one of the manifests (in
>> entailment: [0]) has invalid Turtle syntax, causing my test rig to choke
>> and halt.
>>
>> The problem is the two last prefix declarations in that file (for sd and
>> ent) miss a mandatory trailing dot.
> 
> Hi Jeen,
> I fixed that now.
> 
>> Could someone fix that please? It would be great if I could run my
>> testing directly off the DAWG testset.
>>
>> Also a suggestion: there currently is one single "uber-manifest"[1]
>> which encapsulates all tests for both query and update, it seems. Could
>> I suggest a couple of other manifests alongside that, that group
>> different subsets of all available tests? For example: one manifest for
>> all query evaluation tests, one for all update tests, and one for all
>> (query/update) syntax tests.
> 
> I think that is a good suggestion, but I guess the WG has to decide
> that, so I can't promise anything yet.
> 
> Regards,
> Birte
> 
>> Regards,
>>
>> Jeen
>>
>> [0]
>>
http://www.w3.org/2009/sparql/docs/tests/data-sparql11/entailment/manifest.ttl
>> [1]
>> http://www.w3.org/2009/sparql/docs/tests/data-sparql11/manifest-all.ttl
>>
>>

-- 
PhD Student
IAM Group
Bay 20, Room 4027, Building 32
Electronics & Computer Science
University of Southampton

Received on Thursday, 24 February 2011 11:53:21 UTC