Re: OWL Test Cases

I had a suggestion that we should expand the definition of "logically 
complete", and perhaps require complete consistency checkers to provide 
configurable resource limits. With such a change it would be possible to 
add - "For any given input file there must be some settings of the resource 
limits for which a complete consistency checker returns either Consistent 
or Inconsistent (even if it is after the heat death of the universe)".

I agree with you about the crucial difference between complete and 
incomplete reasoners - we have tried to set it up in that fashion.

Is your problem with the copyright statement that you feel uncomfortable 
*giving* a contribution to the W3C - I have not given this much thought - 
it was basically copied from the RDF Core process. If you can suggest a 
copyright statement that would:
1) permit users to copy
2) permit use in the recommendation
3) keep you happy
I would run it past Dan Connolly to see if there were any procedural 
problems with using it.
I would be relecutant to have the final document littered with differing 
copyright statements - the WD has a copyright statement at the top which 
covers each of the tests as well. These in turn have individual copyright 
statements. The motive for section B.5 is simply to facilitate quality 
control - I automatically check that those exact words are in every file.

We welcome contributions; if you have more than a handful it would make our 
lives much easier if you use the OWL Test Case editor; this generates the 
Manifest files, makes sure the filenaming conventions are followed etc. 
etc. (and adds the copyright statement!)

For details see:
http://lists.w3.org/Archives/Public/www-webont-wg/2003Feb/0267.html

Jim indicated that you want to access only the lite tests ...

The code in Zipper.java in the test case editor should be quite easy to 
modify to make a zip which was only the tests you want.

Jeremy

Bijan Parsia wrote:

> Some comments on: http://www.w3.org/TR/owl-test/ as of 17 February 2003.
> 
> On the one hand (5.2):
>     """A complete OWL Lite consistency checker or a complete OWL DL 
> consistency checker should not return Unknown on the relevant 
> consistency or inconsistency tests.""" (non-normative?)
> 
> on the other (4.4.2):
>     """Note: Complete OWL DL consistency checkers and Complete OWL Lite 
> consistency checkers MAY return Unknown on an OWL DL document or OWL 
> Lite document in the case where a resource limit has been exceeded.""" 
> (normative)
> 
> But this permits  the trivial case of returning "Unknown" for every 
> input. Take putting a timeout in. Just wait 30 seconds (or hours) and 
> then return "Unknown: time limits exceeded".  Or allocate memory until 
> you fill it all up and return unknown.
> 
> More importantly, the difference between a complete and incomplete 
> tableau reasoner (for example) in some cases *isn't* whether they'd run 
> out of resources on the same problem on the same machine, but whether 
> they in principle will terminate. This requires a proof of termination, 
> not a test passing. Though, the obvious, non-pathological differences 
> will be catchable with straightforward tests (e.g., by checking examples 
> where blocking is critical).
> 
> Section B.5 Copyright
> 
>     This makes me very reluctant to submit tests, or suggest to (other 
> non working group) people that they submit tests, especially without a 
> licence condition. But perhaps that's ok :) OTOH, as I develop (and help 
> people develop) reasoners, I expect to generate lots of test cases. 
> Might be nice if this infrastructure (as it develops) can be shared or 
> reused. For example, I've been culling cases from  various papers where 
> the expression lacks finite models (to force termination 
> challenges)...should I submit them?
> 
> Cheers,
> Bijan Parsia.
>

Received on Thursday, 20 February 2003 05:18:55 UTC