exit criteria

Some thoughts about the the rough-draft Exit Criteria [1]:

 * Lest some people think we'll try to cheat on the profiles criterion
   by having 8 DL reasoners :-) I'd add something like:

       Each of these should be a 'native' implementation, taking
       advantage of at least some of the claimed benefits of that
       profile.

 * I suggest we get rid of the "80%" test on OWL 2 Full.  Looking at the
   results from OWL 1 [2] I'm not at all convinced this provided
   evidence of OWL 2 Full being interoperable and useful.  I think I
   could have gotten Surnia up to passing 80%, and Richard Waldinger's
   similar FOL based system [3] might also, but I haven't heard of any
   of them having any traction on the market, even academically.  Pellet
   (which passed 80%) does have traction, but I'm guessing the Pellet
   folks would agree the 80% test isn't useful.

   I'm thinking a better exit criteria for OWL Full might be:

      * Two systems which make use of some OWL 2 Full features (going
        outside of OWL-DL).  Note that we do not expect any practical
        ''complete'' implementations of OWL 2 Full; rather, we expect it
        to be used piecemeal, with systems implementing only the parts
        of OWL 2 Full useful for their application.

 * We're a little too vague about the test suite.  I'd suggest something
   at the top or bottom like:

        Some of these criteria depend on the OWL 2 test suite, which is
        expected to continue to evolve.  For the purposes of these
        criteria, we only consider the "Approved" tests which are not
        "Extra-Credit", and which were approved before some cut-off
        date, to be determined later, some time during CR.

    -- Sandro

 
[1] http://www.w3.org/2007/OWL/wiki/CR_Exit_Criteria
    http://www.w3.org/2007/OWL/wiki/index.php?title=CR_Exit_Criteria&oldid=23906
[2] http://www.w3.org/2003/08/owl-systems/test-results-out#table_1_Approved Full
[3] http://www.w3.org/2001/sw/WebOnt/impls
[4] http://www.ai.sri.com/daml/owl/axiomatic.htm

Received on Wednesday, 20 May 2009 13:41:02 UTC