RE: Open world value tests

-------- Original Message --------
> From: Steve Harris <>
> Date: 24 October 2006 09:34
> 
> On 23 Oct 2006, at 23:29, Eric Prud'hommeaux wrote:
> 
> > > > 
> > > > These tests bring up the issue of how we test extensions. I
think
> > > > all we can do is test the bare language, writing tests that
appear
> > > > to label an extended implementation as "failed".
> > > 
> > > Right - these tests were written to capture "="/"!=" on both known
> > > and unknown types, and what happens when an extension datatype is
> > > known.  Not all the tests will be appropriate in the code DAWG
test
> > > suite. 
> > > 
> > > For testing extensions, how about separating the tests out into a
> > > separate area?  At least, have different manifest files so that an
> > > implementation can pick up the manifests and run the appropriate
> > > ones.
> > 
> > I prefer labeling these tests in the manifest, ala:
> > 
> >         [ mf:name    "date-1" ;
> > 	rdfs:comment	"Added type : xsd:date '='" ;
> >            mf:action
> >               [ qt:query  <date-1.rq> ;
> >                 qt:data   <data-3.ttl> ] ;
> >         mf:result  <date-1-result.srx> ;
> > 	mf:requires xsd:date
> 
> +1

-1

Tests don't have URLs; manifests do have URLs.  Manifests are a unit of
organisation of tests.

Having manifests that an implementer can't just pick up give to their
test framework makes their life harder.  Are they testing with or
without feature X?  And ours too - we will need to provide a set of
tests we wish to get implementation reports on.  Pointing to manifests
by URL is direct and simple.  

Pointing to one manifest that includes the relevant ones is even easier
and that will reduce our workload overall.

Extra vocabulary might also be confusing: an implementer might well say
"my system passed all the tests in manifest X" but not notice that there
were some optional and some base tests.

	Andy

PS Processing that style in SPARQL would require testing for the absense
of a triple pattern involving mf:requires :-)

> 
> - Steve

Received on Tuesday, 24 October 2006 09:07:44 UTC