Re: proposed solution for the RL/Full-testsuite issue

Works for me:-)

Ivan

Michael Schneider wrote:
> Dear all!
> 
> At yesterday's TC there was some larger discussion about my RL/Full testsuite, and we have settled on having further discussion offline. This mail contains a summary of the concerns and a concrete proposal.
> 
> AFAIR, the concerns were:
> 
>   (1) The amount of 700 testcases would change 
>       the current test suite significantly,
>       and are too many to be reviewed visually
> 
>   (2) many of these testcases cannot be checked by 
>       any existing tool
> 
>   (3) only human-reviewed or tool-checked testcases 
>       should be approved by the WG
> 
>   (4) having at the end lots of un-approved testcases 
>       in the Test Wiki is not a good idea 
> 
> In the TC, I have proposed to focus on the subset of the testsuite that is about the OWL 2 RL ruleset. These can be checked against Ivan's RL reasoner. Since Ivan's reasoner and my RL testcases both closely resemble the RL ruleset, bugs in the RL-part of the testsuite should be easy to detect by such a comparison (unless Ivan and I made exactly the same errors independently): If Ivan's reasoner fails on one of my tests, than either his implementation or my testcase must be wrong; this can then be checked visually and fixed. And I have already checked that there exists (at least) one testcase per RL rule (first by design, and later by an additional audit of the whole testsuite).
> 
> Now, this subset for RL still consists of about 300 testcases. But after having looked at them again, I found that the "core" of the RL-part is less than 100 testcases. These are the testcases that cover all the tables in the RL spec except for the datatype table 8. I think, the datatype table should still be covered coarsely, but it's not necessary to check all RL datatypes.
> 
> So here is my proposal: 
> 
>   (a) I will upload to the test Wiki only the "core" of RL testcases 
>       plus the RL testcases for a single datatype (still to be selected). 
>       This will be about 100 testcases. The upload will be completed
>       in the following few days.
> 
>   (b) I will check these RL tests against Ivan's implementation
>       before the next TC (i.e. before 15th of July), and fix bugs 
>       if they occur.
> 
>   (c) The WG can then decide at its next TC to approve the 
>       testcases or not.
> 
>   [(d) People who want a more complete testsuite for RL 
>        (or even beyond) will still be able to download the 
>        larger testsuite from FZI's homepage.]
> 
> Best,
> Michael
> 
> --
> Dipl.-Inform. Michael Schneider
> Research Scientist, Dept. Information Process Engineering (IPE)
> Tel  : +49-721-9654-726
> Fax  : +49-721-9654-727
> Email: michael.schneider@fzi.de
> WWW  : http://www.fzi.de/michael.schneider
> =======================================================================
> FZI Forschungszentrum Informatik an der Universität Karlsruhe
> Haid-und-Neu-Str. 10-14, D-76131 Karlsruhe
> Tel.: +49-721-9654-0, Fax: +49-721-9654-959
> Stiftung des bürgerlichen Rechts, Az 14-0563.1, RP Karlsruhe
> Vorstand: Prof. Dr.-Ing. Rüdiger Dillmann, Dipl. Wi.-Ing. Michael Flor,
> Prof. Dr. Dr. h.c. Wolffried Stucky, Prof. Dr. Rudi Studer
> Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
> =======================================================================
> 
> 

-- 

Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
mobile: +31-641044153
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf