Testing and manifest.rdf

 From Jan.Grant@bristol.ac.uk Tue Sep 25 05:32:31 2001
Status:  U
Return-Path: <Jan.Grant@bristol.ac.uk>
Received: from sisterray.mspring.net ([207.69.231.102])
	by osgood.mail.mindspring.net (Earthlink Mail Service) with 
ESMTP id tr0n9v.jd.37kbi1v
	for <jediofpi@mindspring.com>; Tue, 25 Sep 2001 06:32:31 -0400 (EDT)
Received: from dire.bris.ac.uk (dire.bris.ac.uk [137.222.10.60])
	by sisterray.mspring.net (8.9.3/8.8.6) with ESMTP id GAA34574
	for <aswartz@upclink.com>; Tue, 25 Sep 2001 06:32:30 -0400 (EDT)
Received: from mail.ilrt.bris.ac.uk by dire.bris.ac.uk with SMTP-PRIV
           with ESMTP; Tue, 25 Sep 2001 11:32:17 +0100
Received: from cmjg (helo=localhost)	by mail.ilrt.bris.ac.uk
           with local-esmtp (Exim 3.16 #1)	id 15lpUA-0001jl-00;
           Tue, 25 Sep 2001 11:30:42 +0100
Date: Tue, 25 Sep 2001 11:30:41 +0100 (BST)
 From: Jan Grant <Jan.Grant@bristol.ac.uk>
X-X-Sender:  <cmjg@mail.ilrt.bris.ac.uk>
To: Art Barstow <barstow@w3.org>
cc: Aaron Swartz <aswartz@upclink.com>,
         Dave Beckett <dave.beckett@bristol.ac.uk>,
         Jeremy Carroll <jjc@hplb.hpl.hp.com>,
         "jos.deroo.jd" <jos.deroo.jd@belgium.agfa.com>
Subject: Testing and manifest.rdf
Message-ID: 
<Pine.GSO.4.31.0109251103340.15732-100000@mail.ilrt.bris.ac.uk>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Jan Grant <Jan.Grant@bristol.ac.uk>

As the bunch of people who seemed most interested in the testing aspects
at the last telecon, I thought I'd poll you all about this.

Dave B's got a schema for describing the results of parser tests here:
http://ilrt.org/discovery/2001/03/parser-tests/
(schema is at http://ilrt.org/discovery/2001/03/parser-
tests/schema.rdfs )

but we really need something a little more general.

Tests seem to really be boolean functions of a set of inputs; the inputs
in question are generally pieces of RDF (or ntriples). Tests we've got
thus far are:

positive parser test ( rdf document, ntriples document )
	succeeds if the rdf document parses to produce an rdf model
	equivalent to that specified by the ntriples document

negative parser test ( "nrdf" document )
	succeeds if the document is rejected as being illegal rdf

and suggested tests are also:

positive entailment test ( doc1, doc2 )
	succeeds if the rdf in doc1 entails that in doc2

negative entailment test ( doc1, doc2 )
	succeeds if doc1 and doc2 are legal rdf but doc1 doesn't
	entail doc2.

- I think Jeremy has some other test types he'd ilke to include.
It's worth pointing out that not every input to a test has to be a
document (it could be a literal, such as Dave B's "number of statements
produced").

This is all pretty straightforward, and doesn't have to be overly
complicated; a manifest file can just list a whole bunch of tests one
after another (ie, we don't need to tie them up in a sequence or other
container).

However, there is a problem that crops up principally during parser
tests: some of the RDF/XML parser tests assume (and exercise) a given
base URI for the document (those involving rdf:id in particular). But
we're intending to ship these tests as a ZIP file, so we need to give a
local addres for the test too.

So to describe a single input we need to give:

- type of document (rdf/xml, ntriples, etc)
- base URI of document (if applicable)
- relative path to instantiation of document

And these lead to stylistic issues.

First: we can have the URI of the resource describing an input actually
_be_ the base URI of the document.
Alternatively, we can have some other URI for the document (or keep it
anonymous) and hang a property off it giving the address of the
"canonical instantiation".
In this latter case, there's a secondary issue: should the base URI be a
resource or a literal?

The second issue is how you give a relative path (ie, within the ZIP) to
a local instantiation of an input document. Do you use a file: uri with
a relative (or absolute?) path? Or just a literal?


Weighing against this is the need to keep the manifest file simple and
regular (so it can be parsed only using a simple XML parser, if
required - to bootstrap parser tests).


jan

PS. My feelings are: use anonymous nodes to represent an input document,
and literals to give addresses of instantiations (base URI and local
copy) - but that might be a little contentious.



--
jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/
Tel +44(0)117 9287088 Fax +44 (0)117 9287112 RFC822 jan.grant@bris.ac.uk
Hang on, wasn't he holding a wooden parrot? No! It was a porcelain owl.

Received on Tuesday, 25 September 2001 09:44:08 UTC