- From: Ian Horrocks <ian.horrocks@comlab.ox.ac.uk>
- Date: Mon, 14 Sep 2009 16:02:30 +0100
- To: mak@aifb.uni-karlsruhe.de
- Cc: OWL 2 <public-owl-wg@w3.org>
Excellent -- thanks! Regarding the Unsupported OWL Full tests, IMHO these should not be given approved status. If we were to do so, we would fail to distinguish tests for which we have concrete evidence of correctness---that is, system(s) passing the tests---from those for which we simply don't have any evidence of incorrectness. I think that this is a useful distinction and should be maintained. I don't see why we don't leave the Unsupported OWL Full tests as "proposed". This seems to describe their status rather well -- someone has proposed this test, and we don't yet have sufficient evidence to approve or reject the proposal. Ian On 11 Sep 2009, at 21:45, Markus Krötzsch wrote: > Update: > > * Tests were approved as suggested and the webpage is updated. > * The list of the 6 DL tests that still need more support is at [1]. > * One more test can be "auto-approved": New-Feature- > TopObjectProperty-001 > > Now the majority (82 tests) of our proposed but unsupported tests > belongs to > OWL Full, where all but one were inherited from WebOnt. > > [1] > http://www.w3.org/2007/OWL/wiki/ > Test_Suite_Status#OWL_2_DL_Proposed_Test_Cases > > On Donnerstag, 10. September 2009, Markus Krötzsch wrote: >> Update: >> >> * The official status is called "approved." >> So substitute "accept" by "approve" in my previous mails. >> * New results for Pellet allow two more tests to be approved: >> New-Feature-BottomObjectProperty-001 >> New-Feature-BottomDataProperty-001 >> >> I will send another notice when the test result page has been >> updated with >> the new approved tests. >> >> Markus >> >> On Mittwoch, 9. September 2009, Markus Krötzsch wrote: >>> Of the remaining proposed test cases, there are roughly three kinds: >>> >>> >>> == Almost acceptable DL tests == >>> >>> The following tests already have some support, but not quite >>> enough. I am >>> sure that Pellet or FaCT++ could be made to pass these: >>> >>> * New-Feature-ObjectPropertyChain-BJP-002 >>> this is now passed by HermiT and by REL >>> (but the Exit Criteria require two DL systems, so REL does not >>> suffice) >>> * New-Feature-BottomObjectProperty-001 and >>> New-Feature-BottomDataProperty-001 are passed only by HermiT, but >>> this >>> does not seem to be such a hard reasoning task, after all ;-) >> >> These are now passed by Pellet, so I will change their status to >> approved >> together with the remaining set unless anybody objects. >> >> I forgot some "almost acceptable" DL tests yesterday >> >>> == Almost acceptable OWL Full tests == >>> >>> One passing implementation -- I guess this is enough for OWL Full >>> tests >>> to be acceptable. >>> >>> * WebOnt-I4.6-003 >>> * WebOnt-I4.6-005 >>> * WebOnt-equivalentClass-008 >>> * WebOnt-miscellaneous-302 >>> >>> >>> == Unsupported OWL Full tests == >>> >>> And then there is a large amount of WebOnt tests for which the >>> current >>> OWL Full tools are all incomplete. You can see these tests at >>> [1]. But >>> the Exit Criteria do not require tools to pass all OWL Full >>> tests, so we >>> could also move forward accepting them if nobody objects (they are >>> proposed and WebOnt- accepted, so they have at least seen some human >>> checking and there are some old implementations that have passed >>> some of >>> them). >>> >>> >>> Markus >>> >>> [1] >>> http://www.w3.org/2007/OWL/wiki/ >>> Test_Suite_Status#OWL_2_Full_Proposed_Tes >>> t_ Cases >>> >>> On Mittwoch, 9. September 2009, Ian Horrocks wrote: >>>> Hi Markus, >>>> >>>> Thanks for taking care of this -- the test results look pretty >>>> impressive. >>>> >>>> As far as proposed tests are concerned, we have generally been >>>> accepting any test that passes the basic "eyeball" test and that is >>>> successfully passed by two implementations. If you can send round a >>>> list of proposed test that fall into this category then I think >>>> that >>>> it is reasonable to promote them to approved status unless there >>>> is a >>>> positive objection. >>>> >>>> Thanks, >>>> Ian >>>> >>>> On 9 Sep 2009, at 09:19, Markus Krötzsch wrote: >>>>> After recent updates, the test suite now is in good shape >>>>> regarding >>>>> accepted >>>>> tests [1]. There are still a number of proposed tests which have >>>>> had that >>>>> status for quite a while. I have created an overview of all these >>>>> tests to >>>>> visualize our current coverage: >>>>> >>>>> http://www.w3.org/2007/OWL/wiki/Test_Suite_Status#Proposed_tests >>>>> >>>>> Unfortunately, there are still some errors in the reports (e.g. I >>>>> have positive reports for CEL and FaCT++ for tests that are only >>>>> applicable under >>>>> RDF-based semantics). So some rows in that section are greener >>>>> than >>>>> they >>>>> should be, but most of them should be accurate. >>>>> >>>>> Many of the proposed tests are already covered by two or more >>>>> implementations, >>>>> and thus could probably be accepted easily. Since there is no next >>>>> telecon >>>>> scheduled so far, I hope we could do this via email -- I can >>>>> compile a list of >>>>> tests that seem to be ready. >>>>> >>>>> Regards, >>>>> >>>>> Markus >>>>> >>>>> [1] http://www.w3.org/2007/OWL/wiki/Test_Suite_Status >>>>> >>>>> -- >>>>> Markus Krötzsch >>>>> Institute AIFB, Universität Karlsruhe (TH), 76128 Karlsruhe >>>>> phone +49 (0)721 608 7362 fax +49 (0)721 608 5998 >>>>> mak@aifb.uni-karlsruhe.de www http://korrekt.org >>>>> http://semantic-web-book.org http://semantic-mediawiki.org > > > -- > Markus Krötzsch > Institute AIFB, Universität Karlsruhe (TH), 76128 Karlsruhe > phone +49 (0)721 608 7362 fax +49 (0)721 608 5998 > mak@aifb.uni-karlsruhe.de www http://korrekt.org > http://semantic-web-book.org http://semantic-mediawiki.org
Received on Monday, 14 September 2009 15:03:10 UTC