proposed solution for the RL/Full-testsuite issue

Dear all!

At yesterday's TC there was some larger discussion about my RL/Full testsuite, and we have settled on having further discussion offline. This mail contains a summary of the concerns and a concrete proposal.

AFAIR, the concerns were:

  (1) The amount of 700 testcases would change 
      the current test suite significantly,
      and are too many to be reviewed visually

  (2) many of these testcases cannot be checked by 
      any existing tool

  (3) only human-reviewed or tool-checked testcases 
      should be approved by the WG

  (4) having at the end lots of un-approved testcases 
      in the Test Wiki is not a good idea 

In the TC, I have proposed to focus on the subset of the testsuite that is about the OWL 2 RL ruleset. These can be checked against Ivan's RL reasoner. Since Ivan's reasoner and my RL testcases both closely resemble the RL ruleset, bugs in the RL-part of the testsuite should be easy to detect by such a comparison (unless Ivan and I made exactly the same errors independently): If Ivan's reasoner fails on one of my tests, than either his implementation or my testcase must be wrong; this can then be checked visually and fixed. And I have already checked that there exists (at least) one testcase per RL rule (first by design, and later by an additional audit of the whole testsuite).

Now, this subset for RL still consists of about 300 testcases. But after having looked at them again, I found that the "core" of the RL-part is less than 100 testcases. These are the testcases that cover all the tables in the RL spec except for the datatype table 8. I think, the datatype table should still be covered coarsely, but it's not necessary to check all RL datatypes.

So here is my proposal: 

  (a) I will upload to the test Wiki only the "core" of RL testcases 
      plus the RL testcases for a single datatype (still to be selected). 
      This will be about 100 testcases. The upload will be completed
      in the following few days.

  (b) I will check these RL tests against Ivan's implementation
      before the next TC (i.e. before 15th of July), and fix bugs 
      if they occur.

  (c) The WG can then decide at its next TC to approve the 
      testcases or not.

  [(d) People who want a more complete testsuite for RL 
       (or even beyond) will still be able to download the 
       larger testsuite from FZI's homepage.]

Best,
Michael

--
Dipl.-Inform. Michael Schneider
Research Scientist, Dept. Information Process Engineering (IPE)
Tel  : +49-721-9654-726
Fax  : +49-721-9654-727
Email: michael.schneider@fzi.de
WWW  : http://www.fzi.de/michael.schneider
=======================================================================
FZI Forschungszentrum Informatik an der Universität Karlsruhe
Haid-und-Neu-Str. 10-14, D-76131 Karlsruhe
Tel.: +49-721-9654-0, Fax: +49-721-9654-959
Stiftung des bürgerlichen Rechts, Az 14-0563.1, RP Karlsruhe
Vorstand: Prof. Dr.-Ing. Rüdiger Dillmann, Dipl. Wi.-Ing. Michael Flor,
Prof. Dr. Dr. h.c. Wolffried Stucky, Prof. Dr. Rudi Studer
Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
=======================================================================