Re: test suite maintenance: Syntax vs. Query tests

Jeen Broekstra wrote:
> 
> The current test suite contains test cases for both query evaluation and
> for query syntax testing. These are essentially different ball games.
> 
> To reflect this, Andy has already come up with a different manifest
> format for syntax test cases but many tests still use the 'query'
> manifest format (leaving out the result property).

The original, original design was the outcome of some discussions with 
Alberto, Libby and DanBri.

> 
> Essentially, a syntax test entry as proposed by Andy has three properties:
> 
>   [] rdf:type mf:TestSyntax ;  # to indicate it is a syntax test
>      mf:name "..." ;           # test name, as usual
>      mf:action <query-file.rq> # file with query to be syntax-tested

The format has two levels : a vocabulary for a general test manifest of tests 
with actions and results, then a vocabulary for query tests.

> 
> The 'old' format used for syntax tests is:
> 
>   [] mf:name "...";
>      mf:action [ qt:query <query-file.rq> ;
>                  qt:data <empty.nt> ] ;
> 
> I happen to agree with Andy that his format is more clear and more compact.

It errs on the side of readability - for processing, the code has to know 
what's a blank node and whats a URI node.
Hence some people would like:  mf:action [ qt:query <query-file.rq> ]

> Is everyone ok with me explicitly creating separate suites for
> syntax and query evaluation tests and using Andy's proposed manifest
> format for the syntax tests? I'm happy to do a quick rewrite of the
> relevant manifest (/tests/data/SyntaxFull/manifest.ttl).

I think that would be a good idea - separate vocabulary (subsections of a 
single vocabulary) for syntax tests and query execution tests.

Types help but can detract from readability.  Different properties for syntax 
test query and execute test query (different domains) might give better 
readability (less overhead to the eye).

> 
> Btw, I'm planning to create these separate suites by creating one or two
> 'meta-manifests' that indicate which sets of tests to use for which
> purposes (one for all query-test manifests that are relevant to
> implementations of the latest WD, and perhaps one for all syntax tests).
> This will enable implementors to have a single entry point that tells
> them exactly which tests they should use for their implementation (and
> which they should ignore...).

Good idea.

> 
> Jeen


	Andy

Received on Thursday, 30 March 2006 15:18:25 UTC