ANN: The Fullish Test Suite

Dear all!

You may be interested in a new OWL Full reasoning test suite: the 
Fullish Test Suite! It's a collection of small reasoning examples that 
you can use to either tease your favorite OWL reasoner or to simply get 
a better feeling of what is special about OWL Full. See the full 
announcement below for further information and for a download link.

Cheers,
Michael

=====================================================================
The Fullish Test Suite
2010/11 by Michael Schneider, FZI (schneid@fzi.de)
Access: http://www.fzi.de/downloads/ipe/schneid/testsuite-fullish.zip
=====================================================================

The Fullish Test Suite is a suite of characteristic OWL 2 Full
reasoning test cases. The test cases are not meant to have
important use cases but, rather, their purpose is to demonstrate
different distinguishing semantic aspects of the OWL 2 RDF-Based
Semantics, which are typically not shown by either OWL 2 DL or by
the OWL 2 RL/RDF rules, i.e. each such aspect may be shown by one
or the other but rarely by both entailment regimes. The basic idea
was to create a test suite providing some minimal confidence that
a probed reasoner is really a largely-complete OWL 2 Full reasoner,
if it succeeds on all test cases without faking.

Among the treated aspects of OWL 2 Full are the following: the
existential semantics of blank nodes; data values as individuals;
metamodeling; the use of syntactic and semantic built-in vocabulary
terms as regular entities (sometimes called "syntax reflection");
as well as the strong logic-based semantics of OWL 2, which allows
for sophisticated reasoning based on negation, disjunction, and
universal and existential quantification. Most test cases probe
combinations of several of these aspects. Since it is not always
obvious from a test case that the claimed semantic relationship
really holds, correctness proofs w.r.t. the OWL 2 RDF-Based
Semantics are included for every test case.

Each test case is either an entailment test case consisting of
a premise and a conclusion graph, or an inconsistency test case
consisting of a single graph. There are no test cases for which
the expected result is a non-entailment or a satisfiable ontology.
However, for reasoners implementing RDF-based languages that are
semantically weaker than OWL 2 Full, such as RDFS, some of the
test cases can be used as non-entailment or satisfiability tests,
respectively. The test cases are given in both N3/Turtle syntax
and in the RDF/XML syntax, so they can be used immediately with
most existing Semantic Web reasoners.

In addition, all test cases are encoded in the TPTP language
(http://tptp.org), which is understood by a large number of
available first-order logic theorem provers (FOL-ATPs). Each TPTP
test case encoding consists of two parts: firstly, the encoding
of the actual RDF graphs that represent the test case; and secondly,
the translation of a small subset of OWL 2 RDF-Based semantic
conditions that should, according to the included correctness
proof, be sufficient for a FOL-ATP to succeed on the test case.
Hence, the provided TPTP variants of the test cases can be used
to investigate the appropriateness of off-the-shelf FOL-ATPs for
performing OWL 2 Full reasoning, which can, e.g., be done using
the reasoning service at <http://tptp.org/cgi-bin/SystemOnTPTP>.

Evaluation results from applying this test suite to several
FOL-ATPs, OWL 2 DL reasoners and RDF entailment-rule reasoners
have been reported in the following paper; the test suite is
described in an extended version of the paper:

   Michael Schneider, Geoff Sutcliffe: Reasoning in the OWL 2 Full
   Ontology Language using First-Order Automated Theorem Proving.
   In: Proc. CADE 2011, LNAI, vol. 6803, pp. 446-460. Springer (2011).

   Extended version of paper available at:
     <http://arxiv.org/abs/1108.0155>

   Presentation slides showing evaluation results available at:
  <http://www.slideshare.net/Semantiquele/cade23schneidsutatp4owlfull2011>

-- 
Dipl.-Inform. Michael Schneider
Research Scientist, Information Process Engineering (IPE)
Tel  : +49-721-9654-726
Fax  : +49-721-9654-727
Email: michael.schneider@fzi.de
WWW  : http://www.fzi.de/michael.schneider
==============================================================================
FZI Forschungszentrum Informatik an der Universität Karlsruhe
Haid-und-Neu-Str. 10-14, D-76131 Karlsruhe
Tel.: +49-721-9654-0, Fax: +49-721-9654-959
Stiftung des bürgerlichen Rechts
Stiftung Az: 14-0563.1 Regierungspräsidium Karlsruhe
Vorstand: Dipl. Wi.-Ing. Michael Flor, Prof. Dr. rer. nat. Ralf Reussner,
Prof. Dr. rer. nat. Dr. h.c. Wolffried Stucky, Prof. Dr. rer. nat. Rudi 
Studer
Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
==============================================================================

Received on Wednesday, 17 August 2011 15:31:42 UTC