Re: wording on Unknown returns in Conformance

How does it read if you change "minimally conforming" to "sound" in what I
wrote?One thing that might help is that Peter articulated what the "should
not" implies, adding to his previous email the comment, in the minutes, "some
rationales for should not are in the email I sent out, for OWL Full there
can be others, including known incompleteness (actually it would be possible
for an OWL 2 DL tool to return unknown and say that it just doesn't
implement everything)"

We might include such commentary in the conformance document.

Given the choices -
1) Leave as is
2) Split sound/complete w/wout "must not" wording.
3) Leave as is, but add commentary on what should not means

is any one of these a clear winner in your opinion?

-Alan

On Wed, Sep 24, 2008 at 2:24 PM, Jim Hendler <hendler@cs.rpi.edu> wrote:

> I'd point out to many of the Web Application developers we're trying to
> appeal to, complete is a dirty word -- I think soundness is much more
> important to this crowd than completeness -- consider that many of these
> apps care about getting a few good (correct?) answers quickly rather than
> getting all the answers eventually - I keep saying "think Google" and people
> in the WG keep thinking I'm odd, but out in the real world, every time I say
> this the Web App guys get it -- if Google gave much better suggestions, but
> took two hours (or even two minutes) to do it, they'd be out of business --
> I'm not saying all reasoners should be -- but maybe we simply could define a
> "Sound" reasoner as a level of conformance that makes sense   -JH
> p.s. Yes, I know trivial things can be sound, and all the minimal things
> Peter brings up are bugaboos, but the key is we're trying to fit into a
> large eco-system out there, and different parts have different needs
>
>
> On Sep 24, 2008, at 12:51 PM, Alan Ruttenberg wrote:
>
>
>
> On Wed, Sep 24, 2008 at 12:44 PM, Peter F. Patel-Schneider <
> pfps@research.bell-labs.com> wrote:
>
>> This is not so bad, but it still suffers from the problem that a trivial
>> program can be minimally conforming.
>
>
> Yes. I'm not sure what to do about that, although it might not, in
> practice, be too bad as such a program would be laughed out of the room. I'm
> a little worried that the current OWL-RL statements suffers similarly. OTOH,
> I have thought in the past that the word "complete" ought to be spoken more
> when referring to implementations of reasoners/fragments that are - it's a
> positive word and brings what I think are the right connotations.
>
> -Alan
>
>
>>
>>
>> peter
>>
>>
>> From: "Alan Ruttenberg" <alanruttenberg@gmail.com>
>> Subject: Re: wording on Unknown returns in Conformance
>> Date: Wed, 24 Sep 2008 12:37:49 -0400
>>
>> > I was thinking about something along the lines of (specific words could
>> > change). My changes in italics. There's probably a better word than
>> > "minimally
>> > conforming" but I can't think of one right now.
>> >
>> > An minimally conforming OWL 2 Full entailment checker is an OWL 2
>> > entailment
>> > checker that takes RDF documents as input, and uses the RDF-Based
>> > Semantics
>> > [OWL 2 RDF-Based Semantics]. It must return True only when O1 entails
>> > O2,
>> > and it must return False only when O1 does not entail O2. It may return
>> > Unknown if it is not capable of determining whether an entailment holds
>> > or
>> > not.
>> > A complete OWL 2 Full entailment checker is a minimally conforming OWL 2
>> > Full entailment checker that should not return unknown.
>> >
>> > Then, in section 2.2.1
>> >
>> > It *must* provide a means to determine the semantics it uses (either the
>> > Model-Theoretic Semantics [OWL 2
>> > Semantics<http://www.w3.org/2007/OWL/wiki/Test#ref-owl-2-semantics>]
>> > or the RDF-Based Semantics [OWL 2 RDF-Based
>> > Semantics<http://www.w3.org/2007/OWL/wiki/Test#ref-owl-2-rdf-semantics
>> >]),
>> > and whether it is minimally conformant or complete; for example, in its
>> > supporting documentation.
>> >
>> > Similarly for other profiles.
>> >
>> > I would add a note that at the time of publication it isn't known
>> > whether
>> > complete conformance for OWL Full is possible.
>> >
>> > -Alan
>> >
>> > On Wed, Sep 24, 2008 at 12:26 PM, Peter F. Patel-Schneider <
>> > pfps@research.bell-labs.com> wrote:
>> > > Huh?  What is this, and how would it be done.
>> > >
>> > > peter
>> > >
>> > > From: "Alan Ruttenberg" <alanruttenberg@gmail.com>
>> > > Subject: Re: wording on Unknown returns in Conformance
>> > > Date: Wed, 24 Sep 2008 12:17:07 -0400
>> > >
>> > >> I had thought that it might be worth distinguishing distinct levels
>> > of
>> > >> conformance - complete versus incomplete. Do you think that would be
>> > a
>> > >> good idea? It bothers me a bit that conformance as specified for OWL
>> > >> Full, as stated now, is not known to be possible.
>> > >> -Alan
>> > >>
>> > >>
>> > >> On Wed, Sep 24, 2008 at 10:04 AM, Peter F. Patel-Schneider
>> > >> <pfps@research.bell-labs.com> wrote:
>> > >> >
>> > >> > There have been some comments on the "should not" wording in the
>> > >> > conformance part of the Test and Conformance document.
>> > >> >
>> > >> > The current wording includes
>> > >> >
>> > >> > An OWL 2 Full entailment checker is an OWL 2 entailment checker
>> > that
>> > >> > takes RDF documents as input, and uses the RDF-Based Semantics [OWL
>> > 2
>> > >> > RDF-Based Semantics]. It MUST return True only when O1 entails O2,
>> > and
>> > >> > it MUST return False only when O1 does not entail O2. It SHOULD NOT
>> > >> >  return Unknown.
>> > >> >
>> > >> > Without the last sentence, a trivial checker, i.e., one that always
>> > >> > returned "Unknown" would be just as good an OWL 2 Full entailment
>> > >> > checker as one that tried hard.
>> > >> >
>> > >> > Even worse, if the last sentence was removed from
>> > >> >
>> > >> > An OWL 2 DL entailment checker is an OWL 2 entailment checker that
>> > >> >  takes OWL 2 DL ontology documents as input, and uses the Model
>> > >> > Theoretic Semantics [OWL 2 Semantics]. It MUST return True only
>> > when
>> > >> > O1 entails O2, and it MUST return False only when O1 does not
>> > entail
>> > >> >  O2. It SHOULD NOT return Unknown.
>> > >> >
>> > >> > then a trivial checker would be just as good as a complete reasoner
>> > for
>> > >> > OWL 2 DL.
>> > >> >
>> > >> >
>> > >> > I thus feel that there needs to be some wording in the conformance
>> > >> > document to show that trivial checkers, or unnecessarily incomplete
>> > >> > checkers, are not as good as ones that return "Unknown" in fewer
>> > cases.
>> > >> >
>> > >> > Remember that "should not" is not the same as "must not".  A
>> > checker
>> > >> > could return "Unknown" if
>> > >> > 1/ it ran out of resources (memory, time, etc.); or
>> > >> > 2/ it is an incomplete reasoner (for OWL 2 Full, for example, or
>> > even
>> > >> >   for OWL 2 DL).
>> > >> > The above reasons (or others) could be used by entailment checkers
>> > to
>> > >> > provide a justification for "Unknown" answers.  I feel, however,
>> > that
>> > >> > this is outside the scope of the specification.
>> > >> >
>> > >> > Perhaps it would be useful to add some wording on justifying
>> > "Unknown"
>> > >> > to the document, but I think that most of this is implied by the
>> > use of
>> > >> > "should not".
>> > >> >
>> > >> > peter
>> > >> >
>> > >> >
>> > >
>>
>
>
>  "If we knew what we were doing, it wouldn't be called research, would
> it?." - Albert Einstein
>
> Prof James Hendler http://www.cs.rpi.edu/~hendler
> Tetherless World Constellation Chair
> Computer Science Dept
> Rensselaer Polytechnic Institute, Troy NY 12180
>
>
>
>
>