RE: Testing the (RL) testing...

>-----Original Message-----
>From: public-owl-wg-request@w3.org [mailto:public-owl-wg-request@w3.org]
>On Behalf Of Ian Horrocks
>Sent: Wednesday, June 03, 2009 5:21 PM
>To: Mike Smith
>Cc: mak@aifb.uni-karlsruhe.de; Michael Schneider; Ivan Herman; W3C OWL
>Working Group
>Subject: Re: Testing the (RL) testing...
>
>An RL implementation using the RDF-based semantics is a sound but
>incomplete OWL Full reasoner. So, it makes perfect sense to run such
>a system against *all* tests where the RDF-based semantics is
>applicable. What could/should be usefully distinguished, however, are
>positive-entailment/subsumption/unsatisfiability tests where
>implementations based on the RL rule set will return True. Conforming
>systems *must* pass these tests.

In my test suite, those test cases that are positive entailment or
inconsistency tests w.r.t. the RL ruleset will explicitly be marked 
as such (by whichever means that will be most appropriate for this 
in the test wiki). An implementation claiming conformance to the 
RL ruleset should then succeed on all these test.

>Unfortunately, identifying these tests will not, in general, be
>completely trivial -- we either need a "trusted" implementation with
>no additional features/rules, or we need a trusted pair of eyeballs.

I am using a fairly cautious process to build my suite. But errors
can never be avoided, of course. Well, I am doing my best.

>However, even identifying some tests that are in this category and
>some "Full" tests that are outside of this category would be a useful
>addition to the test set. Note that conforming RL systems are *not*
>obliged to "fail" the latter kind of test - they are simply not
>*obliged* to pass them.

It will be interesting to apply an RL-plus-a-bit 
implementation (for example, a future Jena reasoner that builds on 
top of the current Jena OWL implementation, and also covers all of 
the RL rules) to the *whole* "rdfbased" test suite, since this will 
give some idea "how far" such an implementation is away from the 
whole of OWL 2 Full (provided that my test suite will really give
appropriate coverage of the whole of OWL 2 Full, let's see...).

But enough advertising! :) There's still some work to do for me.

>Ian

Cheers,
Michael

--
Dipl.-Inform. Michael Schneider
Research Scientist, Dept. Information Process Engineering (IPE)
Tel  : +49-721-9654-726
Fax  : +49-721-9654-727
Email: michael.schneider@fzi.de
WWW  : http://www.fzi.de/michael.schneider
=======================================================================
FZI Forschungszentrum Informatik an der Universität Karlsruhe
Haid-und-Neu-Str. 10-14, D-76131 Karlsruhe
Tel.: +49-721-9654-0, Fax: +49-721-9654-959
Stiftung des bürgerlichen Rechts, Az 14-0563.1, RP Karlsruhe
Vorstand: Prof. Dr.-Ing. Rüdiger Dillmann, Dipl. Wi.-Ing. Michael Flor,
Prof. Dr. Dr. h.c. Wolffried Stucky, Prof. Dr. Rudi Studer
Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
=======================================================================