RE: Testing the (RL) testing...

>-----Original Message-----
>From: public-owl-wg-request@w3.org [mailto:public-owl-wg-request@w3.org]
>On Behalf Of Mike Smith
>Sent: Wednesday, June 03, 2009 3:15 PM
>To: mak@aifb.uni-karlsruhe.de
>Cc: Michael Schneider; Ivan Herman; W3C OWL Working Group
>Subject: Re: Testing the (RL) testing...
>
>On Wed, Jun 3, 2009 at 03:00, Markus Krötzsch <mak@aifb.uni-
>karlsruhe.de> wrote:
>> On Mittwoch, 3. Juni 2009, Michael Schneider wrote:
>
>>> I guess, by "rl=In RL" it is basically meant that the ontologies
>>> conform /syntactically/ to OWL 2 RL, which means they have to satisfy
>>> the syntactic restrictions of the DL-ish variant of OWL 2 RL.
>
>Correct, the field is intended to match this definition from
>Conformance.
>
>"An OWL 2 RL ontology document is an OWL 2 DL ontology document where
>the corresponding instance of the OWL 2 ontology class satisfies the
>definition of an OWL 2 RL ontology given in the OWL 2 Profiles
>specification. "
>
>
>>> But why don't we simply re-interpret this for the case of OWL 2 RL,
>>> so that the above combination (semantics=full, dl=no, rl=yes) means
>>> that the testcase can safely be consumed by systems implementing the
>>> RL-ruleset?
>
>Can we identify such cases?  
>As I understand it, we only make
>assertions about completeness of the rules in Theorem PR1, which
>requires the input ontologies to be syntactically RL.

Do you suggest to restrict OWL 2 RL test cases to only the fraction of the
language that is covered by TH1? This is a very tiny fraction, basically
only allowing for certain (not even all) kinds of assertions, and leaving
out all "higher" language features of RL. I do not see why we want to have a
huge part of the RL language being uncovered by test cases and, more
specifically, I do not see why the particular theoretic result of TH1 should
at all have any relevance for the question which part of the language should
be covered by test cases and which not.
 
Further, the theorem hasn't any relevance for the scenario I am talking
about here. The theorem can be seen to be about that subset of the RL
syntactic fragment where implementations based on the RL-ruleset are
complete (and sound) w.r.t. the direct semantics. I am completely
indifferent about this point in this discussion. I am interested here in the
completeness of claimed RL implementations w.r.t. the RL ruleset itself, and
when applied to arbitrary RDF triple stores. 

For example, I might be interested whether Oracle's RL implementation, which
will be a black box to me (unless I get Zhe to leak some information about
its implementation :-)), gives me all the results that the RL ruleset
promises, for ontologies using /any/ of the RL language feature (and not
just property and class assertions, and equality), and without being
restricted to ontologies conforming to the syntactic RL profile. I won't use
just those restricted (and pretty complicated, I never get it right which
expressions are allowed where in an RL ontology) kinds of KBs with Oracle's
RDBMS. I'd rather wanted to use Oracle as a generic RDF triple store and let
its built-in RL reasoner produce (at least) any conclusion from the stored
RDF that the RL ruleset provides. 

And there are actually many conclusions that can be drawn from unrestricted
RDF graphs that do not satisfy the syntactic restrictions of OWL 2 DL, let
alone RL. For example, the RL/rules allow for the following entailment

  owl:sameAs rdfs:subPropertyOf ex:p .
  ex:x rdf:type owl:Thing .
  ------------------------------------
  ex:x ex:p ex:x .

which is certainly outside DL. (You can see this inference being produced by
Ivan's RL service at <http://www.ivan-herman.net/Misc/2008/owlrl/>.) This
example may be contrived, but an RL entailment checker that implements (at
least) the RL ruleset can be expected to accept the above entailment query
as valid input and to give the answer "true". And I see no reason why not to
write test cases that probe implementation for such expected conclusions. If
an implementation claims to be an "RL/ruleset compatible" reasoner, then
such test cases help me to find out that this is really the case.

My upcoming "rdfbased" test suite contains many test cases that happen to be
RL-ruleset valid entailments and, although most of them are less contrived
than the example above, they very often fall out of DL (and, in particular,
out of the RL syntactic fragment), since DL-compliance hasn't been a
criterion when producing the test cases. And, beyond basic language feature
coverage, there will even be a handful of "RDF-ish" test cases that match
the RL ruleset, but which are inherently outside DL and cannot even be
"DL-ized".

>>> This should then also show up in the Test-Wiki, as
>>> something like:
>>>
>>>   Syntactic Species/Profile    OWL 2 Full (RL=ok)
>>
>> Right, the application logic of those templates has been designed when
>we were
>> still using "DL" and "Full" even for semantics, and when all profiles
>were
>> assumed to be sublanguages of DL. This should obviously be changed now
>-- I
>> will take care of this. Maybe we should also check if there are any
>> ramifications for the Conformance document (but I believe that this
>has been
>> updated many times since these changes happened).
>
>
>It would be helpful if someone can describe how we identify test cases
>where the input ontologies are not syntactically RL but for which
>RL=ok.

Of course, in principle one would need to distinguish between syntactic RL
conformance on the one hand (and use the currently existing "rl" flag for
just this purpose), and on the other hand testcases being targeted to probe
implementations w.r.t. the RL ruleset (on unrestricted RDF): the "OWL 2 RL
entailment checkers that apply the RDF-based semantics..." (note that OWL 2
RL entailment checkers "take OWL 2 Full ontology documents as input"). But,
unless we have an additional flag for this, I am happy to interpret the
combination of "semantics=rdf-based-only" and "rl=true" to have just the
meaning of that additional flag. It's a hack, though... 

>Thanks,
>--
>Mike Smith
>
>Clark & Parsia

Best,
Michael

--
Dipl.-Inform. Michael Schneider
Research Scientist, Dept. Information Process Engineering (IPE)
Tel  : +49-721-9654-726
Fax  : +49-721-9654-727
Email: michael.schneider@fzi.de
WWW  : http://www.fzi.de/michael.schneider
=======================================================================
FZI Forschungszentrum Informatik an der Universität Karlsruhe
Haid-und-Neu-Str. 10-14, D-76131 Karlsruhe
Tel.: +49-721-9654-0, Fax: +49-721-9654-959
Stiftung des bürgerlichen Rechts, Az 14-0563.1, RP Karlsruhe
Vorstand: Prof. Dr.-Ing. Rüdiger Dillmann, Dipl. Wi.-Ing. Michael Flor,
Prof. Dr. Dr. h.c. Wolffried Stucky, Prof. Dr. Rudi Studer
Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
=======================================================================