OWL Test Cases

Some comments on: http://www.w3.org/TR/owl-test/ as of 17 February 2003.

On the one hand (5.2):
	"""A complete OWL Lite consistency checker or a complete OWL DL 
consistency checker should not return Unknown on the relevant 
consistency or inconsistency tests.""" (non-normative?)

on the other (4.4.2):
	"""Note: Complete OWL DL consistency checkers and Complete OWL Lite 
consistency checkers MAY return Unknown on an OWL DL document or OWL 
Lite document in the case where a resource limit has been exceeded.""" 
(normative)

But this permits  the trivial case of returning "Unknown" for every 
input. Take putting a timeout in. Just wait 30 seconds (or hours) and 
then return "Unknown: time limits exceeded".  Or allocate memory until 
you fill it all up and return unknown.

More importantly, the difference between a complete and incomplete 
tableau reasoner (for example) in some cases *isn't* whether they'd run 
out of resources on the same problem on the same machine, but whether 
they in principle will terminate. This requires a proof of termination, 
not a test passing. Though, the obvious, non-pathological differences 
will be catchable with straightforward tests (e.g., by checking examples 
where blocking is critical).

Section B.5 Copyright

	This makes me very reluctant to submit tests, or suggest to (other 
non working group) people that they submit tests, especially without a 
licence condition. But perhaps that's ok :) OTOH, as I develop (and help 
people develop) reasoners, I expect to generate lots of test cases. 
Might be nice if this infrastructure (as it develops) can be shared or 
reused. For example, I've been culling cases from  various papers where 
the expression lacks finite models (to force termination 
challenges)...should I submit them?

Cheers,
Bijan Parsia.

Received on Thursday, 20 February 2003 12:41:06 UTC