Re: test suite maintenance

Jeen Broekstra wrote:
> On Wed, 2006-11-01 at 10:57 +0000, Seaborne, Andy wrote:
>>
>> Jeen Broekstra wrote:
>>> We're currently testing our new SPARQL parser/engine and ran into an
>>> issue with the DAWG syntax test cases.
>>>
>>> According to
>>> http://www.w3.org/2001/sw/DataAccess/tests/data/SyntaxFull/manifest.ttl , specifically DAWG-approved tests syntax-bnodes-03.rq and syntax-bnodes-04.rq, it is legal syntax to have blank node labels in the predicate position. (either [] or _:a).
>> http://www.w3.org/2001/sw/DataAccess/tests/data/SyntaxDev/
>>
>> is the working copy based on the current grammar. (Dev, not Full)
>>
>> Full is (or rather "was") the version of the first full version of the syntax 
>> that the WG approved.
> 
> Ah ok. I've switched to using SyntaxDev.
> 
> I see you are using an idea we had earlier of creating a 'super
> manifest' that can include multiple other manifests. Currently the super
> manifest uses a jena-specific namespace,

Yes, they do - DAWG has not agreed a change to the manifest format so I didn't 
want to invade a namespace with new vocabulary when I don't own that 
namespace.  Isn't that the right thing to do?

Some details:
http://lists.w3.org/Archives/Public/public-rdf-dawg/2005JulSep/0246

> however. To make it more
> generic (and also a bit easier to use for us) I would like to make the
> following changes:
> 
>  - the 'include' property and the 'TestSyntax' class are added to 
>    the DAWG test case vocabulary; the jena-specific namespace can
>    then be removed.

That looses the fact that sub-manifests are ordered in the file.  We already 
have ordered tests in manifests using lists.

>  - the range of the 'include' property is redefined as a single 
>    Manifest URL (in your current version it's a collection which is 
>    harder to process, for me at least), so the super-manifest will
>    contain a 'include' property for each manifest.
> 
> If nobody objects I will implement and check in these changes. 

See also:
http://lists.w3.org/Archives/Public/public-rdf-dawg/2005JulSep/0326

The SyntaxDev area is a copy of the grammar development tests. I keep the 
masters elsewhere (they may be changing as work on the grammar proceeds) and I 
add and update them live.

Should I send you any new tests I write?

The master copies are in:
http://jena.cvs.sourceforge.net/jena/ARQ/testing/ARQ/Syntax/?pathrev=MAIN

specifically, the .sh scripts which write the test files and also create a 
manifest that matches the current tests.  With so many tests, it's easy to add 
a test but fail to add it to the manifest.  The .sh files bring that all into 
one place.

As this isn't working, I'll stop check-pointing what I currently use and 
assume that if anyone is interested in a live set, they can take it from ARQ 
and risk that it is in-flux occasionally for a day or two.  That better for you?


Please could we agree what the test suite format is.  Giving individual tests 
each a URL in the manifest would be good.  I have quite a lot of queries and 
manifests now and I don't want to make changes to format, and to the code to 
process it, if it's going to change again.  The test processing code is quite 
old so a good rewrite will do it no harm - but munging the input manifests 
each time is just churn for me.

 Andy

> 
> Jeen

Received on Wednesday, 1 November 2006 14:23:29 UTC