Re: wording on Unknown returns in Conformance

I'd point out to many of the Web Application developers we're trying  
to appeal to, complete is a dirty word -- I think soundness is much  
more important to this crowd than completeness -- consider that many  
of these apps care about getting a few good (correct?) answers quickly  
rather than getting all the answers eventually - I keep saying "think  
Google" and people in the WG keep thinking I'm odd, but out in the  
real world, every time I say this the Web App guys get it -- if Google  
gave much better suggestions, but took two hours (or even two minutes)  
to do it, they'd be out of business -- I'm not saying all reasoners  
should be -- but maybe we simply could define a "Sound" reasoner as a  
level of conformance that makes sense
  -JH
p.s. Yes, I know trivial things can be sound, and all the minimal  
things Peter brings up are bugaboos, but the key is we're trying to  
fit into a large eco-system out there, and different parts have  
different needs


On Sep 24, 2008, at 12:51 PM, Alan Ruttenberg wrote:

>
>
> On Wed, Sep 24, 2008 at 12:44 PM, Peter F. Patel-Schneider <pfps@research.bell-labs.com 
> > wrote:
> This is not so bad, but it still suffers from the problem that a  
> trivial
> program can be minimally conforming.
>
> Yes. I'm not sure what to do about that, although it might not, in  
> practice, be too bad as such a program would be laughed out of the  
> room. I'm a little worried that the current OWL-RL statements  
> suffers similarly. OTOH, I have thought in the past that the word  
> "complete" ought to be spoken more when referring to implementations  
> of reasoners/fragments that are - it's a positive word and brings  
> what I think are the right connotations.
>
> -Alan
>
>
>
> peter
>
>
> From: "Alan Ruttenberg" <alanruttenberg@gmail.com>
> Subject: Re: wording on Unknown returns in Conformance
> Date: Wed, 24 Sep 2008 12:37:49 -0400
>
> > I was thinking about something along the lines of (specific words  
> could
> > change). My changes in italics. There's probably a better word than
> > "minimally
> > conforming" but I can't think of one right now.
> >
> > An minimally conforming OWL 2 Full entailment checker is an OWL 2
> > entailment
> > checker that takes RDF documents as input, and uses the RDF-Based
> > Semantics
> > [OWL 2 RDF-Based Semantics]. It must return True only when O1  
> entails
> > O2,
> > and it must return False only when O1 does not entail O2. It may  
> return
> > Unknown if it is not capable of determining whether an entailment  
> holds
> > or
> > not.
> > A complete OWL 2 Full entailment checker is a minimally conforming  
> OWL 2
> > Full entailment checker that should not return unknown.
> >
> > Then, in section 2.2.1
> >
> > It *must* provide a means to determine the semantics it uses  
> (either the
> > Model-Theoretic Semantics [OWL 2
> > Semantics<http://www.w3.org/2007/OWL/wiki/Test#ref-owl-2-semantics>]
> > or the RDF-Based Semantics [OWL 2 RDF-Based
> > Semantics<http://www.w3.org/2007/OWL/wiki/Test#ref-owl-2-rdf-semantics 
> >]),
> > and whether it is minimally conformant or complete; for example,  
> in its
> > supporting documentation.
> >
> > Similarly for other profiles.
> >
> > I would add a note that at the time of publication it isn't known
> > whether
> > complete conformance for OWL Full is possible.
> >
> > -Alan
> >
> > On Wed, Sep 24, 2008 at 12:26 PM, Peter F. Patel-Schneider <
> > pfps@research.bell-labs.com> wrote:
> > > Huh?  What is this, and how would it be done.
> > >
> > > peter
> > >
> > > From: "Alan Ruttenberg" <alanruttenberg@gmail.com>
> > > Subject: Re: wording on Unknown returns in Conformance
> > > Date: Wed, 24 Sep 2008 12:17:07 -0400
> > >
> > >> I had thought that it might be worth distinguishing distinct  
> levels
> > of
> > >> conformance - complete versus incomplete. Do you think that  
> would be
> > a
> > >> good idea? It bothers me a bit that conformance as specified  
> for OWL
> > >> Full, as stated now, is not known to be possible.
> > >> -Alan
> > >>
> > >>
> > >> On Wed, Sep 24, 2008 at 10:04 AM, Peter F. Patel-Schneider
> > >> <pfps@research.bell-labs.com> wrote:
> > >> >
> > >> > There have been some comments on the "should not" wording in  
> the
> > >> > conformance part of the Test and Conformance document.
> > >> >
> > >> > The current wording includes
> > >> >
> > >> > An OWL 2 Full entailment checker is an OWL 2 entailment checker
> > that
> > >> > takes RDF documents as input, and uses the RDF-Based  
> Semantics [OWL
> > 2
> > >> > RDF-Based Semantics]. It MUST return True only when O1  
> entails O2,
> > and
> > >> > it MUST return False only when O1 does not entail O2. It  
> SHOULD NOT
> > >> >  return Unknown.
> > >> >
> > >> > Without the last sentence, a trivial checker, i.e., one that  
> always
> > >> > returned "Unknown" would be just as good an OWL 2 Full  
> entailment
> > >> > checker as one that tried hard.
> > >> >
> > >> > Even worse, if the last sentence was removed from
> > >> >
> > >> > An OWL 2 DL entailment checker is an OWL 2 entailment checker  
> that
> > >> >  takes OWL 2 DL ontology documents as input, and uses the Model
> > >> > Theoretic Semantics [OWL 2 Semantics]. It MUST return True only
> > when
> > >> > O1 entails O2, and it MUST return False only when O1 does not
> > entail
> > >> >  O2. It SHOULD NOT return Unknown.
> > >> >
> > >> > then a trivial checker would be just as good as a complete  
> reasoner
> > for
> > >> > OWL 2 DL.
> > >> >
> > >> >
> > >> > I thus feel that there needs to be some wording in the  
> conformance
> > >> > document to show that trivial checkers, or unnecessarily  
> incomplete
> > >> > checkers, are not as good as ones that return "Unknown" in  
> fewer
> > cases.
> > >> >
> > >> > Remember that "should not" is not the same as "must not".  A
> > checker
> > >> > could return "Unknown" if
> > >> > 1/ it ran out of resources (memory, time, etc.); or
> > >> > 2/ it is an incomplete reasoner (for OWL 2 Full, for example,  
> or
> > even
> > >> >   for OWL 2 DL).
> > >> > The above reasons (or others) could be used by entailment  
> checkers
> > to
> > >> > provide a justification for "Unknown" answers.  I feel,  
> however,
> > that
> > >> > this is outside the scope of the specification.
> > >> >
> > >> > Perhaps it would be useful to add some wording on justifying
> > "Unknown"
> > >> > to the document, but I think that most of this is implied by  
> the
> > use of
> > >> > "should not".
> > >> >
> > >> > peter
> > >> >
> > >> >
> > >
>

"If we knew what we were doing, it wouldn't be called research, would  
it?." - Albert Einstein

Prof James Hendler				http://www.cs.rpi.edu/~hendler
Tetherless World Constellation Chair
Computer Science Dept
Rensselaer Polytechnic Institute, Troy NY 12180