Re: test suite reorg: added evaluation tests

Lee Feigenbaum wrote:
> Andy Seaborne wrote on 02/16/2007 10:04:45 AM:
>> Jeen Broekstra wrote:
>>> I have checked in four additional sets of query evaluation test cases 
> in
>>> the data-r2 directory:
>>>
>>> - 
> http://www.w3.org/2001/sw/DataAccess/tests/data-r2/bnode-coreference/
>>>   (1 test)
>>> - http://www.w3.org/2001/sw/DataAccess/tests/data-r2/bound/
>>>   (1 test)
>>> - http://www.w3.org/2001/sw/DataAccess/tests/data-r2/optional/
>>>   (3 tests)
>>> - http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-builtin/
>>>   (17 tests)
>>  >
>>> As far as I have been able to determine, all these tests are up to 
> date
>>> with the current query spec. Additional eyeballs are invited to 
> correct  :)
>> expr-builtin/manifest.ttl uses <#dawg-langMatches-1> in the entries list 
> but 
>> the file later has <#dawg-LangMatches-1> (change of case of the "L" of 
> lang).
>> Tweaking that, ARQ passes these tests.
>>
> 
> My test harness can't handle the five tests in this batch that specify 
> results in .srx (SPARQL XML results) format. Our test README ( 
> http://www.w3.org/2001/sw/DataAccess/tests/README.html ) suggests that a 
> test result given in a test manifest "is a resource that refers to an RDF 
> graph." Wihle I can clearly (and easily) convert the .srx format into our 
> RDF test result vocabulary (or even check the results directly from the 
> .srx format), I -- as an implementor -- was surprised to find that some of 
> the test results are given in this format. Perhaps we can either update 
> the README to point this out or update these tests?

My preference is to update the README (which was written before srx files were 
defined).  I can imagine systems that only provide the srx output (and only 
support SELECT queries).

FWIW, ARQ has a conversion tool, arq.rset, that reads and writes the various 
formats.

I now recall - which is why those tests has ORDER BY in them because it made 
tetsign against srx files very simple at a time when we were all getting stuff 
implemented.

> 
> Also: the manifest.ttl for expr-builtin pointed to the wrong query and 
> results for #dawg-lang-3. I've checked in the fix.

Cool - I updated data-r2 and all the tests (26) pass for manifest-evaulation.ttl

> 
> I pass all the tests except for the 5 with .srx results, and I've 
> eyeballed those to my satisfaction.
> 
> Lee
> 

	Andy

Received on Tuesday, 20 February 2007 14:08:46 UTC