W3C home > Mailing lists > Public > www-webont-wg@w3.org > September 2003

Re: OWL Test Results page, built from RDF

From: Sandro Hawke <sandro@w3.org>
Date: Fri, 05 Sep 2003 01:03:20 -0400
Message-Id: <200309050503.h8553K5W013759@roke.hawke.org>
To: "Jos De_Roo" <jos.deroo@agfa.com>
Cc: "Dan Connolly <connolly" <connolly@w3.org>, www-webont-wg@w3.org


> > http://lists.w3.org/Archives/Public/www-archive/2003Sep/att-0005/_xxx.rdf
> 
> http://www.agfa.com/w3c/temp/owl-result.rdf

> just that I don't know what to use instead of
> http://www.w3.org/2002/03owlt/resultsOntology#FailingRun
> to indicate a failing test (which we still have ;-))

Sorry, I had a bug in handling FailingRuns.  Should be fixed now.

I've added the above two URLs to the generation of the static page
[1], do it shows Euler, Surnia, and OWLP.  I also added the CR
manifest file, and use it to provide status and descriptions of the
tests.  For the program/CGI, it's just another input which you can
remove or replace with your custom Manifest.

Note how I handled OWLP, with a raft of URI-less tests like "Syntactic
Level Test for complementOf-001".  To pass such a test, the tested
system must correctly identify the language level of ALL the input
files for the associated test.   (If someone has a better idea how to
handle this, let me know.)

My big dilemma right now is how we should report Consistency and
Negative Entialment test results for incomplete reasoners.  I'm not
comfortable with saying "Pass" when you just time out, but as Jos
pointed out, the fact that you were not able to find an inconsistency
is still useful.  Maybe something like "Partial", which would be
considered better than "Incomplete" but still not as good as "Pass".
This would allow an OWL Full implementation to, in theory, do okay
(Pass/Partial) on every test.   Basically, "Incomplete" would be
counted as "Partial" for certain types of reasoners on certain types
of tests.   Maybe it should just be "Good Incomplete" and "Bad
Incomplete"... -- but that distinction can be made in my code, as long
as its told which kind of reasoner is involved.

     -- sandro
Received on Friday, 5 September 2003 01:05:14 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 7 December 2009 10:58:02 GMT