- From: Bijan Parsia <bparsia@isr.umd.edu>
- Date: Wed, 19 Feb 2003 19:19:06 -0500 (EST)
- To: public-webont-comments@w3.org
- Cc: "Jeremy J. Carroll" <jjc@hpl.hp.com>, Jos De Roo <jos.deroo@agfa.com>
Some comments on: http://www.w3.org/TR/owl-test/ as of 17 February 2003. On the one hand (5.2): """A complete OWL Lite consistency checker or a complete OWL DL consistency checker should not return Unknown on the relevant consistency or inconsistency tests.""" (non-normative?) on the other (4.4.2): """Note: Complete OWL DL consistency checkers and Complete OWL Lite consistency checkers MAY return Unknown on an OWL DL document or OWL Lite document in the case where a resource limit has been exceeded.""" (normative) But this permits the trivial case of returning "Unknown" for every input. Take putting a timeout in. Just wait 30 seconds (or hours) and then return "Unknown: time limits exceeded". Or allocate memory until you fill it all up and return unknown. More importantly, the difference between a complete and incomplete tableau reasoner (for example) in some cases *isn't* whether they'd run out of resources on the same problem on the same machine, but whether they in principle will terminate. This requires a proof of termination, not a test passing. Though, the obvious, non-pathological differences will be catchable with straightforward tests (e.g., by checking examples where blocking is critical). Section B.5 Copyright This makes me very reluctant to submit tests, or suggest to (other non working group) people that they submit tests, especially without a licence condition. But perhaps that's ok :) OTOH, as I develop (and help people develop) reasoners, I expect to generate lots of test cases. Might be nice if this infrastructure (as it develops) can be shared or reused. For example, I've been culling cases from various papers where the expression lacks finite models (to force termination challenges)...should I submit them? Cheers, Bijan Parsia.
Received on Thursday, 20 February 2003 12:41:06 UTC