Re: TEST review copy

"Accepted" means I will make the change to the editors draft

>>at the time of its publication. ther documents may supersede
>>
>                                   ^ "O"
> 


Accepted.


> 
>>We seek reports from implementators concerning both errors in the
>>tests and successful execution of these tests, both proposed and
>>approved. The tests are still in development. and are being
>>changed. during the last call review period, up to the publication of
>>the corresponding Proposed Recommendation. 
>>
> 
> Some typos, but also it's pretty, um, awkward.  How about:
> 
> We plan to keep improving these tests until they are published as a
> Proposed Recommendation.  Please let us know (at the comments address
> below) if you find any errors.  We would also appreciate a note
> describing your experience with these tests if you have developed an
> OWL implementation.
> 


Accepted - note this is in the SOTD which is owned by W3C i.e. Dan and you. 
The words currently there are merely my suggestions to you.


> 
>>to which this document is subsiduary. 
>>
>                             ^^^^^^^^^^  subsidiary
> 


Accepted.


> Does that mean that if the documents disagree, the other one is
> correct?


Yes.


> 
> 
>>However, the tests do not constitute a conformance test suite
>>for OWL. 
>>
> 
> Clarify?  Perhaps by adding something like: "... since they are silent
> on several important issues.  This document cannot be considered a
> complete specification of OWL."
> 
> 

Accepted.


>> for the larger tests, by reference to the test Web site.
>>
> 
> Has that practice been affirmed by the Comm team?
> 


This is old text which should have gone.
The current situation is that:
- the normative compound document includes everything
- the extra large single document includes everything
- the medium single document includes none of the tests
- the large single document includes some of the tests (this used to be the 
normative version)

I will propose some new simplified text, that refers to the situation in 
the normative version only. (Probably delete ", either explicitly or, ... 
Web site.").
(Note also a text variability mess: The L version says "This recommendation 
track document" no link, the others say "The recommendation track document" 
with a (correct) hyperlink to './'") I will get rid of the magic text and 
make it all say "The <a href="./">recommendation track docoument</a>").





> 
>>2.1.  Normative Status
>>Of the deliverables the only normative tests are those included in
>>this document. All other deliverables, are informative.  
>>
>                                        ^ extra comma
> 


Accepted.


> This seems to disagree with what I quote above, "The recommendation
> track document, which normatively includes the tests, either
> explicitly or, for the larger tests, by reference to the test Web
> site." 
> 


This section is correct.


> 
>>Each test consists of either one or more RDF/XML documents and a
>>
>                         ^^^^^^  extra word
> 


Accepted.


> 
>>The datatypes xsd:integer, xsd:string from [XML Schema Datatypes]
>>are not indicated, even when used or required.  
>>
> 
>    + "because their theories are a part of OWL" (or something)
> 


Accepted - I will work on the (or something).


> 
>>the rest have names beginning supportNNN.rdf.
>>
> 
> names _like_ supportNNN.rdf ?   
> 



"_like_ supportNNN-A.rdf" (I think, need to check).

Accepted.


> 
>>4.2.1. Syntax Checker
>>
> 
> It seems odd to say each program outputs one word and may also output
> warnings.   Describe stdout & stderr?  :-)
>


Howabout s/outputs/returns/

then "returns" seems a clearly different verb from "give a warning"

similarly s/outputs/returns/ in 4.2.2 consistency checker.

 
> 
> 
>>An OWL consistency checker MUST return Consistent only when the
>>input document is consistent and Inconsistent only when the input
>>document is not consistent (this property is usually called
>>soundness).  
>>
> 
> It took me a while to connect "this property" with its referrent.  How
> about:
> 
>    An OWL consistency checker MUST be <em>sound</em>: it MUST return
>    Consistent only when the input document is consistent and
>    Inconsistent only when the input document is not consistent 
> 
>


Accepted.

 
> 
>> otherwise it is complete
>>
>                    ^ you mean "incomplete"
>


Accepted.

 
> 
>>  It has been shown that ...
>>
> 
> This needs a footnote/link, but I don't know where the proof is.
> 
> For instance, I'm having a hard time believing OWL full doesn't have a
> FOL axiomatization.... 
>


That does not give you complete reasoning I believe.
There is a horrocks paper that shows this result, I will add a reference.

 
> 
>>Every OWL Lite consistency checker can be trivially transformed into
>>an OWL Full consistency checker.  
>>
> 
> This is just talking about outputting "Unknown" as mentioned two lines
> down, right?  If so, then please drop this sentence from this
> paragraph, where it is horribly confusing.
>


Accepted. (This sentence is there because *I* didn't much like the group 
decision to have three levels for things that are essentially the same).

 
> 
>>Note: Complete OWL DL consistency checkers and Complete OWL Lite
>>consistency checkers MAY return Unknown on an OWL DL document or OWL
>>Lite document in the case where a resource limit has been exceeded.  
>>
> 
> How is a testing framework going to live with this?  I'd think they'd
> need to return "ResourceLimitExceeded".
> 


Not accepted.

Rationale:
The difference between a decidable and an undecidable unknown is more 
sophisticated than can be shown with this. An algorithm that does not 
provably terminate, can be arbitrarily interrupted, just as one that does 
provably terminate. Hence the difference between a complete and an 
incomplete reasoner is not one that can be demonstrated with a test 
harness, and we shouldn't try.

These definitions (syntax checker, and consistency checker), are not, in my 
mind, trying to define a particularly useful API, merely a minimal set of 
functionality. So in practice, I would expect an OWL Consistency Checker to 
differentiate the various reasons for "Unknown" but I don't believe we 
should try and enumerate them, or mandate such a practice).


> 
>>A complete OWL Lite consistency checker or a complete OWL DL
>>consistency checker should not return Unknown  
>>
>                       ^^^^^^^^^^  MUST NOT
> 


This is a substantive change, requiring WG decision, I will start a new thread.


> 
> 
>>These tests are ones that are either known from the literature (for
>>instance, from [Heinsohn et al.]), or from test suites contributed
>>by Network Inference.  
>>
>      ^^^^^^^^^^^^^^^^^     make that a link, or something?
>


Accepted. www.networkinference.com

            
> [ both the Normative and Informative sections ]
>

(There is only one source text)


 
> 
>> 2. Bag ID.
>>
> 
> I think you can remove that bit, now.
> 
>

Accepted.

 
>>any that do gain extra credit. 
>>
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   this bit made me laugh, 
> it's probably not necessary or appropriate.
>



I will accept this if you insist.

Why not - the title for the section was proposed, I think jokingly by Mike 
Dean, in the January f2f; however for non-US readers the phrase "extra 
credit" needs just a little explanation, which is this phrase. Note: this 
section will never be normative.

 
> But I love the test itself, and can't wait to try to pass it!
> 
> ...
>


It would be good to populate the section more.

 
> The test website itself needs some cleanup, but I guess that's a
> different job.
> 


There is the editors note asking for feedback on this issue.
It has been there since the first draft, this is the first feedback I have 
received. I am inclined to leave the note in and see whether anybody else 
wants this. (Trying to avoid work).

Jeremy

Received on Monday, 12 May 2003 04:46:05 UTC