wording on Unknown returns in Conformance

There have been some comments on the "should not" wording in the
conformance part of the Test and Conformance document.

The current wording includes

  An OWL 2 Full entailment checker is an OWL 2 entailment checker that
  takes RDF documents as input, and uses the RDF-Based Semantics [OWL 2
  RDF-Based Semantics]. It MUST return True only when O1 entails O2, and
  it MUST return False only when O1 does not entail O2. It SHOULD NOT
  return Unknown.

Without the last sentence, a trivial checker, i.e., one that always
returned "Unknown" would be just as good an OWL 2 Full entailment
checker as one that tried hard.

Even worse, if the last sentence was removed from 

  An OWL 2 DL entailment checker is an OWL 2 entailment checker that
  takes OWL 2 DL ontology documents as input, and uses the Model
  Theoretic Semantics [OWL 2 Semantics]. It MUST return True only when
  O1 entails O2, and it MUST return False only when O1 does not entail
  O2. It SHOULD NOT return Unknown.  

then a trivial checker would be just as good as a complete reasoner for
OWL 2 DL.


I thus feel that there needs to be some wording in the conformance
document to show that trivial checkers, or unnecessarily incomplete
checkers, are not as good as ones that return "Unknown" in fewer cases.

Remember that "should not" is not the same as "must not".  A checker
could return "Unknown" if
1/ it ran out of resources (memory, time, etc.); or
2/ it is an incomplete reasoner (for OWL 2 Full, for example, or even
   for OWL 2 DL).
The above reasons (or others) could be used by entailment checkers to
provide a justification for "Unknown" answers.  I feel, however, that
this is outside the scope of the specification.

Perhaps it would be useful to add some wording on justifying "Unknown"
to the document, but I think that most of this is implied by the use of
"should not".

peter

Received on Wednesday, 24 September 2008 14:05:23 UTC