RE: complete checkers should or MUST pass tests

> The change seemed editorial to me.  There are six sentences in section
> 5 ("Testing an OWL Implementation (Informative)"); five of them are
> of the form "x must y", and this last one says "x should not y".  I
> didn't see any reason why it should be weaker than the others, but as
> a reader the fact that it was weaker (after 5 musts) stood out.

These "musts" and "should" were derived (by me) from the normative "MUSTS"
above.

A complete OWL consisitency checker MUST be logically complete, and so MUST
eventually pass all the consisitency and inconsistency tests. But that
eventually MAY be after the heat death of the universe, and so I weakened
the last one from a "must" to a "should" since events after the end of the
universe are not very interesting.

>
> To go against the "should" would be to go against the normative
> definition of "complete", as far as I can tell, so a "must" here is
> logicically redundant.  But the absense of "must" is suggestive of
> some other state of affairs.
>
> Meanwhile, I didn't notice that section 5 was only informative.  That
> seems odd for a section of "musts".  Is that because this information
> is all logically redundant (and its bad practice to risk conflict by
> having redundant normative text) or because we specifically don't want
> to say that OWL software (of the appropriate category) has to pass the
> appropriate tests.

It is logically redundant; also without a software category to run the
entailment tests I didn't want to go into too much detail as to what you
MUST (or even must) do with them. In practice, I believe that the entailment
tests are useful; however I went to some effort to turn some (non)entailment
tests into (in)consistency tests for this reason.

Here is old text that I dropped with the introduction of the consistency
checker conformance statement:
http://www.w3.org/TR/2002/WD-owl-test-20021024/#runningReasoner

>  (It would be logically redundant if we said
> elsewhere the tests had to be passed; a quick look does not turn up
> such text, but one could read it into the fact that the tests
> themselves are labeled Normative.)

We say that all possible tests have to be passed; we can hence deduce that a
specific selection of tests (the ones endorsed by the WG or any other) has
to be passed.

>
> (Meanwhile, I also noticed the odd lack of any category of software
> related to the entailments test, but I guess it's too late in the
> process to do anything about that.)
>

At the January f2f we invented the conformance statements (with a degree of
angst; if I remember. The morning session ended somewhat abruptly; I took a
long lunch break during which the rest of the group sorted it - rather
better I suspect than if I had been present).

It was felt more appropriate to concentrate on document characteristics.
The entailment tests require more than one document.

====

An interesting background picture is found in:

http://kogs-www.informatik.uni-hamburg.de/~haarslev/publications/m4m2.pdf

figure 3 on page 10.

While the details of the paper are not relevant, the point is that without a
certain optimization presented in the paper you get performance as shown by
the exponential lines; with the optimization you get constant performance.

While I am not sure whether the specific techniques in that paper are
needed, I note that they would be very helpful in some of the tests such as
found in:

http://www.w3.org/2002/03owlt/editors-draft/draft/proposed-dl-900-arith#prop
osed-dl-900-arith
C.3.6 Extended Caridnality Testing

in particalar test 907

http://www.w3.org/2002/03owlt/editors-draft/draft/proposed-dl-900-arith#dl-9
00-arith-907

(with a typo in the description)
This is intended to be on the right in the figure.

Replacing "should" with "MUST" would either:
- limit our ability to approve tests requiring certain optimizations
or
- require that complete reasoners used all the optimizations tested

Either would have the desirable effect of making it clearer to users of
complete OWL consistency checkers what they might expect in terms of
practical behaviour. I personally would vote for this change; but it is a
substantive change.

Jeremy

Received on Monday, 12 May 2003 07:26:40 UTC