W3C home > Mailing lists > Public > www-webont-wg@w3.org > September 2003

Re: OWL Test Results page, built from RDF

From: Jos De_Roo <jos.deroo@agfa.com>
Date: Sun, 7 Sep 2003 13:33:25 +0200
To: Ian Horrocks <horrocks@cs.man.ac.uk>
Cc: Sandro Hawke <sandro@w3.org>, www-webont-wg@w3.org, www-webont-wg-request@w3.org, jjc@hpl.hp.com
Message-ID: <OFBCA6887D.8D19F4BC-ONC1256D9A.003F1755-C1256D9A.003F87D8@agfa.be>

Many thanks for the clarification Ian.
I believe this is very constructive
(in both senses ;-))

Jos De Roo, AGFA http://www.agfa.com/w3c/jdroo/

                      Ian Horrocks                                                                                                     
                      <horrocks@cs.man.        To:       Jos De_Roo/AMDUS/MOR/Agfa-NV/BE/BAYER@AGFA                                    
                      ac.uk>                   cc:       "Jeremy Carroll <jjc", Sandro Hawke <sandro@w3.org>, www-webont-wg@w3.org     
                      Sent by:                 Subject:  Re: OWL Test Results page, built from RDF                                     
                      2003-09-07 11:59                                                                                                 
                      Please respond to                                                                                                
                      Ian Horrocks                                                                                                     

On September 5, Jos De_Roo writes:
> > > My big dilemma right now is how we should report Consistency and
> > > Negative Entialment test results for incomplete reasoners.  I'm not
> > > comfortable with saying "Pass" when you just time out, but as Jos
> > > pointed out, the fact that you were not able to find an inconsistency
> > > is still useful.  Maybe something like "Partial", which would be
> > > considered better than "Incomplete" but still not as good as "Pass".
> > > This would allow an OWL Full implementation to, in theory, do okay
> > > (Pass/Partial) on every test.   Basically, "Incomplete" would be
> > > counted as "Partial" for certain types of reasoners on certain types
> > > of tests.   Maybe it should just be "Good Incomplete" and "Bad
> > > Incomplete"... -- but that distinction can be made in my code, as
> > > as its told which kind of reasoner is involved.
> >
> >
> > If we use the term 'incomplete' it is not perjorative, merely a
> > description.
> > However, given that even WG members cannot emotionally buy that, using
> > 'partial' instead is better.
> I was wrong - IncompleteRun is indeed a good idea for
> saying "Pass" when you just time out (my confusion
> was that I thought we then couldn't get a FailingRun for
> a Consistency and Negative Entialment test, but we can).
> I will try to update my testresults that way.

A sound but incomplete reasoner can return 3 different results -
"yes", "no" and "don't know". We clearly need to distinguish failure
in the sense that a yes/no answer was returned but was incorrect from
the case where a don't-know answer is returned. Reporting something
like "incomplete" for the don't-know case seems reasonable and, as
was pointed out by Jeremy, is not pejorative.

I don't believe that it is either desirable or sensible for the
results to distinguish good/bad incompleteness. Bad incompleteness is
unsoundness and can simply be reported as "fail".


> --
> Jos De Roo, AGFA http://www.agfa.com/w3c/jdroo/
Received on Sunday, 7 September 2003 07:34:14 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:56:55 UTC