Re: Test suite status

Update:

* New results for FaCT++ have arrived; webpage updated,
* We now have two passing implementations for test New-Feature-DataQCR-001.
  I will approve this test if there are no objections.
* The list of the 5 DL tests that still need more support is at [1].

[1] 
http://www.w3.org/2007/OWL/wiki/Test_Suite_Status#OWL_2_DL_Proposed_Test_Cases


On Freitag, 11. September 2009, Markus Krötzsch wrote:
> Update:
>
> * Tests were approved as suggested and the webpage is updated.
> * The list of the 6 DL tests that still need more support is at [1].
> * One more test can be "auto-approved": New-Feature-TopObjectProperty-001
>
> Now the majority (82 tests) of our proposed but unsupported tests belongs
> to OWL Full, where all but one were inherited from WebOnt.
>
> [1]
> http://www.w3.org/2007/OWL/wiki/Test_Suite_Status#OWL_2_DL_Proposed_Test_Ca
>ses
>
> On Donnerstag, 10. September 2009, Markus Krötzsch wrote:
> > Update:
> >
> > * The official status is called "approved."
> >   So substitute "accept" by "approve" in my previous mails.
> > * New results for Pellet allow two more tests to be approved:
> >   New-Feature-BottomObjectProperty-001
> >   New-Feature-BottomDataProperty-001
> >
> > I will send another notice when the test result page has been updated
> > with the new approved tests.
> >
> > Markus
> >
> > On Mittwoch, 9. September 2009, Markus Krötzsch wrote:
> > > Of the remaining proposed test cases, there are roughly three kinds:
> > >
> > >
> > > == Almost acceptable DL tests ==
> > >
> > > The following tests already have some support, but not quite enough. I
> > > am sure that Pellet or FaCT++ could be made to pass these:
> > >
> > > * New-Feature-ObjectPropertyChain-BJP-002
> > >   this is now passed by HermiT and by REL
> > >   (but the Exit Criteria require two DL systems, so REL does not
> > > suffice) * New-Feature-BottomObjectProperty-001 and
> > > New-Feature-BottomDataProperty-001 are passed only by HermiT, but this
> > > does not seem to be such a hard reasoning task, after all ;-)
> >
> > These are now passed by Pellet, so I will change their status to approved
> > together with the remaining set unless anybody objects.
> >
> > I forgot some "almost acceptable" DL tests yesterday
> >
> > > == Almost acceptable OWL Full tests ==
> > >
> > > One passing implementation -- I guess this is enough for OWL Full tests
> > > to be acceptable.
> > >
> > > * WebOnt-I4.6-003
> > > * WebOnt-I4.6-005
> > > * WebOnt-equivalentClass-008
> > > * WebOnt-miscellaneous-302
> > >
> > >
> > > == Unsupported OWL Full tests ==
> > >
> > > And then there is a large amount of WebOnt tests for which the current
> > > OWL Full tools are all incomplete. You can see these tests at [1]. But
> > > the Exit Criteria do not require tools to pass all OWL Full tests, so
> > > we could also move forward accepting them if nobody objects (they are
> > > proposed and WebOnt- accepted, so they have at least seen some human
> > > checking and there are some old implementations that have passed some
> > > of them).
> > >
> > >
> > > Markus
> > >
> > > [1]
> > > http://www.w3.org/2007/OWL/wiki/Test_Suite_Status#OWL_2_Full_Proposed_T
> > >es t_ Cases
> > >
> > > On Mittwoch, 9. September 2009, Ian Horrocks wrote:
> > > > Hi Markus,
> > > >
> > > > Thanks for taking care of this -- the test results look pretty
> > > > impressive.
> > > >
> > > > As far as proposed tests are concerned, we have generally been
> > > > accepting any test that passes the basic "eyeball" test and that is
> > > > successfully passed by two implementations. If you can send round a
> > > > list of proposed test that fall into this category then I think that
> > > > it is reasonable to promote them to approved status unless there is a
> > > > positive objection.
> > > >
> > > > Thanks,
> > > > Ian
> > > >
> > > > On 9 Sep 2009, at 09:19, Markus Krötzsch wrote:
> > > > > After recent updates, the test suite now is in good shape regarding
> > > > > accepted
> > > > > tests [1]. There are still a number of proposed tests which have
> > > > > had that
> > > > > status for quite a while. I have created an overview of all these
> > > > > tests to
> > > > > visualize our current coverage:
> > > > >
> > > > > http://www.w3.org/2007/OWL/wiki/Test_Suite_Status#Proposed_tests
> > > > >
> > > > > Unfortunately, there are still some errors in the reports (e.g. I
> > > > > have positive reports for CEL and FaCT++ for tests that are only
> > > > > applicable under
> > > > > RDF-based semantics). So some rows in that section are greener than
> > > > > they
> > > > > should be, but most of them should be accurate.
> > > > >
> > > > > Many of the proposed tests are already covered by two or more
> > > > > implementations,
> > > > > and thus could probably be accepted easily. Since there is no next
> > > > > telecon
> > > > > scheduled so far, I hope we could do this via email -- I can
> > > > > compile a list of
> > > > > tests that seem to be ready.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Markus
> > > > >
> > > > > [1] http://www.w3.org/2007/OWL/wiki/Test_Suite_Status
> > > > >
> > > > > --
> > > > > Markus Krötzsch
> > > > > Institute AIFB, Universität Karlsruhe (TH), 76128 Karlsruhe
> > > > > phone +49 (0)721 608 7362           fax +49 (0)721 608 5998
> > > > > mak@aifb.uni-karlsruhe.de           www  http://korrekt.org
> > > > > http://semantic-web-book.org  http://semantic-mediawiki.org


-- 
Markus Krötzsch
Institute AIFB, Universität Karlsruhe (TH), 76128 Karlsruhe
phone +49 (0)721 608 7362           fax +49 (0)721 608 5998
mak@aifb.uni-karlsruhe.de           www  http://korrekt.org
http://semantic-web-book.org  http://semantic-mediawiki.org