TEST: proposal for test case structure

Summary:
- copy RDF Core
- use directories per issue and per feature
- use boring file names that indicate test type and a number
- use RDF/XML as normative, N-triple informative
- have at least entailment, non-entailment, consistency, inconsistency and bad
test types.
- use manifest file to list tests and their status
- any WG member can create tests, but status is "PROPOSED"
- delete/move/rename current test dir contents

I suggest we aim to copy RDF Core except where they appear to have made mistakes
or where WebOnt goals differ.

This involves.

1: Directories of directories of test cases.

I suggest the following:

http://www.w3.org/2002/03owlt/issues/

is a directory.

For each issue with tests we construct a named subdirectory
e.g. for 5.1

http://www.w3.org/2002/03owlt/issues/I5.1-Uniform-treatment-of-literal-data-valu
es

We also have

http://www.w3.org/2002/03owlt/features/

as a directory.
And for each language feature we have a sub-directory.

(RDF Core only has issue driven tests - the structure above differs from theirs
to explicitly allow for other tests)

2: File locations

I suggest test files should only appear in leaf directories.

3: File types

I suggests all normative tests should be in RDF/XML.
Each of these should have a non-normative N-triple analogue, produced
automatically.
(Or perhaps the method of production is the converse!)
There should be no n3 in the test cases directory. (There currently is).

4: Test types

We have currently approved tests that are:
- entailment tests
and
- illegal use of OWL namespace

I expect that we will have consistency tests as well (this document is/is not
self-consistent). These could be regarded as a special case of entailment.

We may need other test types.

4.1 Entailment test

A positive entailment test consists of two documents. One is the premises, the
other is the conclusions.
The conclusions follow from the premises.

A negative entailment test  consists of two documents. One is the premises, the
other is the conclusions.
The conclusions do not follow from the premises.

4.2 Illegal document test.

An illegal document test consists of one document.
This document is not legal.
A comment indicates the nature of the illegality.

4.3 Consistency test

A positive consistency test consists of one document.
The document is consistent.

A positive consistency test consists of one document.
The document is inconsistent.

5: Test file naming

In RDF Core the names of the test files *do not* convey enough information to
execute the test. (But nearly) A test manifest file is needed.

In RDF core nearly all the files are named

test0NN.rdf and test0NN.nt

where 0NN is a three digit number.

IMO, this has not worked well for entailment tests because of confusion about
premises and conclusions.

I suggest that for each test type there should be naming conventions for those
files.

e.g.

5.1 Entailment test

A positive entailment test consists of two documents.
premises001.rdf
conclusions001.rdf

If the converse entailment holds this is only recorded in the manifest.

A negative entailment test consists of two documents.
premises001.rdf
nonconclusions001.rdf


5.2 Illegal document test.

An illegal document test consists of one document.
bad001.rdf

5.3 Consistency test

A positive consistency test consists of one document.
consistent001.rdf

A positive consistency test consists of one document.
inconsistent001.rdf


6: Manifest file

Each directory has a manifest file describing the tests in it.
RDF core has defined a schema. I suggest we use that with extensions.
We may wish to make a richer description of the schema using OWL.

The manifest file may represent tests using the rdf files which are not apparant
from the naming conventions. The most common will be two-way entailment
(equivalent documents); it might be worth having a naming convention for those.

7: File creating and modification

Proposed tests should be added by any WG member to the appropriate subdirectory.
These should be added to the manifest with status="PROPOSED".

tests can be updated by their creator or someone acting on behalf of the WG, or
any member of the TEST subgroup making editorial changes.



=====

An issue that appears is that the current test case area is already populated
with files that do not adhere to these proposed conventions.

Choices are:
- move/rename/delete those files
- start with a fresh empty test area

Received on Tuesday, 16 July 2002 07:43:04 UTC