Re: accuracy check on OWL-DL reasoners

On 8 Nov 2007, at 10:16, <gstoil@image.ece.ntua.gr> wrote:

> Hi Bijan and all,
>
> Do you also count "bugs" (in the sense of incorrect or inaccurate
> implementation of the theoretical algorithm rather than exceptions  
> and related
> stuff)

I don't know what distinction you are trying to raise. If you  
implement nominals using the disjoint classes approximation, then I  
wouldn't call you complete. If you implement the SHOIN tableau with  
the NN Rule, then, assuming it isn't utterly ridiculous, I'd count it  
as complete.

I would also count it as complete if it implemented a (reasonable)  
hard coded resource bound. Say, 1 year and 100 terabytes of memory  
(or even something tied to pointer size on a particular architecture).

> as cases of incompleteness of an implemented system?

Every software system has bugs. There is not a bright light when a  
buggy implementation ceases to be an implementation.

Similarly, naive implementations that are clearly hopeless in all  
cases probably shouldn't "count" (at least for some purposes).

However, if Pellet counts as complete as of 6 months ago, then I  
would easily class it as complete as of at least a year and a half  
and more likely of two years ago. It makes a good faith effort to  
implement correctly and in a reasonably optimized way a decision  
procedure for SHOIN and for SROIQ.

How this bears on your decisions about whether to include a feature  
in whatever part of the RIF is under discussion is outside my ken.

Cheers,
Bijan.

Received on Thursday, 8 November 2007 10:42:24 UTC