Re: Action-317 Review of Conformance and Test Cases

On 1 Apr 2009, at 07:16, Alan Ruttenberg wrote:

> Here is my review of the Conformance and Test Cases document.
> http://www.w3.org/2007/OWL/wiki/index.php? 
> title=Conformance_and_Test_Cases&oldid=20353
>
> Section 1.
>
> nor does it constitute a ^comprehensive^ conformance test suite
> Is it the case that returning a false on a positive entailment test
> indicates nonconformance?

The distinction between exhaustive and comprehensive is debatable I  
think -- probably neither is perfect. I changed it to "complete",  
which I think is closer to what is intended (the test suite can be  
thought of as a sound but incomplete procedure for deciding non- 
conformance), and mentioned the "sound" case as suggested.

>
> Section 2.1.2
>
> Regarding:
> "Note that OWL 2 Profiles may support only a reduced set of
> datatypes. This is, however, a syntactic condition that must be met by
> documents in order to fall within the relevant profile, and the
> semantic conditions on the supported datatypes are unchanged, i.e.,
> they are defined by a (possibly extended) OWL 2 Datatype map."
>
>
> I didn't understand this. Does this mean that the profiles can have an
> extended datatype map?

Currently, profiles say which OWL datatypes must *not* be used, and  
also describe the necessary condition of the set of datatypes (the  
intersection of the value spaces of any set of these datatypes is  
either empty or infinite) [NB: check on what RL now says]. Thus  
profiles could, in principle, have an extended datatype may provided  
that the relevant condition is satisfied.

What the above statement says is that we don't need to use a reduced  
datatype map when specifying semantic conformance conditions for  
profiles, because defining the semantics of extra "unused" datatypes  
doesn't hurt us -- in the same way that using an extended datatype  
map doesn't hurt us in the case of DL and Full. I added a sentence  
that I hope makes this clearer.


>
> Section 2.2
>
> I worry that the use of Ont(d) to denote either a structure or a graph
> is confusing and that it might be better to give these distinct names.

In some sense it is *meant* to be confusing, in that it should  
(usually) be irrelevant which one we are talking about, and we want  
to make single statements that are true in any case.


>
> Wording:
>
> "Given an ontology document d in the RDF/XML serialization, for a tool
> applying the Direct Semantics, Ont(d) denotes the ontology structure
> obtained from d via the canonical parsing process as defined in the
> OWL 2 Syntax specification [OWL 2 Specification] and the procedure for
> mapping from RDF graphs to the structural specification described in
> the OWL 2 Mapping to RDF Graphs [OWL 2 Mapping to RDF Graphs];"
>
> ->
>
> "Given an ontology document d in the RDF/XML serialization, for a tool
> applying the Direct Semantics, Ont(d) denotes the ontology structure
> obtained by applying the canonical parsing process as defined in the
> OWL 2 Syntax specification [OWL 2 Specification] to d using, in steps
> 2.2 and 3.3, the procedure for mapping from RDF graphs to the
> structural specification described in the OWL 2 Mapping to RDF Graphs
> [OWL 2 Mapping to RDF Graphs];"

I changed it slightly to "... by applying to d ..."

>
> In the below, the juxtaposition of normative "must" with the loose
> "similar" is odd. Rephrase?
>
> "The conformance conditions related to entailment checking and query
> answering are defined below. Other OWL 2 tools must satisfy similar
> conditions. In particular, they must be consistent with the Direct
> Semantics [OWL 2 Direct Semantics] and/or the RDF-Based Semantics [OWL
> 2 RDF-Based Semantics]."
>
> Section 2.2
>
> I would like to see a requirement that tools can operate in a 'strict'
> mode, in which the datatype map is limited to be the smaller of the
> OWL 2 Datatype Map or the set of datatypes that a profile is
> syntactically restricted to, in order to support applications where
> interoperability between OWL implementations is a high priority.

Nothing prevents tools operating in this mode. Making it mandatory  
that tools can be *forced* to operate in this mode would, I think,  
require some kind of WG resolution.

>
> Section 2.2.1
>
> Wording:
>
> "Additionally, an OWL 2 entailment checker... must provide a means to
> determine the datatypes supported by its datatype map, and any limits
> it has on datatype lexical forms — for example by listing them in its
> supporting documentation (see Section 4 of the OWL 2 Syntax
> specification [OWL 2 Specification]); "
>
> It would be the provider of the entailment checker that listed the
> datatype map etc, rather than the entailment checker itself.
>
> In the below, I think FO should be expanded to "first order".
>
> ... "denotes the FO theory corresponding to Ont(di) in which triples
> are represented using the T predicate — that is, T(s, p, o) represents
> an RDF triple with the subject s, predicate p, and the object o."
>
> Section 3.1.1
>
> "Syntactic tests can be applied to tools that process OWL 2 ontology
> documents, or that transform between various syntactic forms of OWL
> 2. These modes of operation are not covered by any conformance
> requirement, but syntactic tests may still be useful in tool
> development."
>
> I don't understand the comment about there not being a conformance
> requirement. Section 2.1.1 lists syntactic conformance requirements
> and we have, in the RDF Mapping, "for any OWL 2 DL ontology O, let G =
> T(O) be the RDF graph obtained by transforming O as specified in
> Section 2, and let OG be the OWL 2 DL ontology obtained by applying
> the reverse transformation from Section 3 to G; then, O and OG are
> logically equivalent — that is, they have exactly the same set of
> models."

But we don't say anywhere what would constitute a conforming syntax  
checker or converter.

>
> Section 3.1.1.2
>
> Do we want to say that translations preserve structure, or preserve
> semantics? Currently it says structure which may be too strong.

It doesn't say that tools *must* preserve structure -- in fact as I  
mentioned above we don't define any conformance conditions for this  
kind of tool. Tests of this kind are probably useful and appropriate,  
however, as most tools will intend to preserve structure and it will  
be useful for implementers to have tests for this (e.g., for testing  
the GRDDL transform).

>
> Are not all the ontologies in a translation test normative?

I suppose that Manchester Syntax ontologies at least would not be  
normative.

>
> Section 3.1.2
>
> "Semantic tests specify one or more OWL 2 ontology documents and check
> semantic conditions defined with respect to abstract structures
> obtained from the ontology documents, typically via a parsing
> process."
>
> What would an atypical case be?

This is a repeat of what is said in 2.2. If we change it here we  
should change it there too. I could easily be persuaded.

>
> Section 3.1.2.4
>
> Why is O(in) not included in the test ontology rather than only given
> in an appendix?

Because it isn't really part of the test -- it is an inconsistent  
ontology that could be used to transform the inconsistency test into  
an entailment test. I reworded this slightly (it shouldn't say *the*  
inconsistent ontology).

>
> I would move "Details on the changes of the test case format as
> compared to WebOnt are found in Section 3.5." to immediately after the
> sentence
>
> "Existing test ontologies, such as the ones used by the WebOnt working
> group [OWL Test Cases], have not been crafted this way and do not meet
> the above requirements."
>
> "it is, however, not necessary to compute entailments of this ontology
> in order to use the provided test case documents"
>
> I'm not clear on why this is the case. Strictly based on looking at
> the ontology there are cases where it looks like entailments are
> needed (for example :semantics property of
> :alternativeSemanticsTest). Will these entailments not be necessary
> because they will be stated explicitly in the test documents?

Not sure. Mike and/or Markus can no doubt clarify. In fact, I'm going  
to give up at this point and defer to Mike/Marcus for the rest.

Ian


>
> Section 3.2.2
>
> There is no class for syntactic translation tests.
>
> "In practice, semantic tests will indeed have only one premise,
> conclusion, or non-conclusion, but for convenience each of those may
> be provided in multiple syntactic forms. This is the reason why the
> above assertions do not require exact cardinalities."
>
> Do we expect there to be at most one in each syntax? If so, you can
> use qualified max cardinality here.
>
> Section 3.2.3
>
>
> "The individual :FUNCTIONAL indicates that all functional syntax input
> ontologies are normative"
> Here and elsewhere the scope of "all" wasn't clear.
>
> The choice of ObjectOneOf( :RDFXML :FUNCTIONAL :OWLXML ) unecessarily
> limits extensibility of the (future contributed) tests to other
> syntaxes. Better to use a class and subclasses so that other syntaxes
> can be added later.
>
> Declaration(Class(:OWL2Syntax))
> SubClassOf(:RDFXML :OWL2Syntax)
> ...
> ObjectPropertyRange( :normativeSyntax :OWL2Syntax)
>
> Here and elsewhere property ranges are given, but not domains
> (:TestCase). Any reason not to?
>
> Section 3.2.6
>
> Wording
> "The following axiom reflects the fact that OWL 2 EL and OWL 2 QL are
> syntactic profiles of OWL 2 DL."
>
> profile != subset in our spec. Use "subset"? "restriction" or just say
> "are also" OWL 2 DL.
>
> Also, 2.1.1 says: "An OWL 2 RL ontology document is an OWL 2 DL
> ontology document" so shouldn't RL be mentioned here too?
>
> Section 3.2.8
>
> Do we really want to keep rejected test cases? Why?  For test cases
> contributed after the end of the working group, does it make sense to
> consider these "proposed" given that there may be no one to propose
> them to? Should we add another value "contributed"?
>
> Section 3.2.9
>
> :identifier, :creator, :description seem like dc:identifier,
> dc:creator, rdfs:comment.
> Should we reuse these previously defined properties?
> Allowing a rdfs:label seems like a good idea.
>
> It may be worth doing an audit of the test ontology to see if there
> are other opportunities to use already defined properties.
>
> I wonder if :identifier should be further restricted by a pattern
> facet, as the logical definition doesn't follow the textual one.
>
> Values for creator and description should perhaps be rdf:text rather
> than xsd:string to allow for (future) translations and non-english
> contributions.
>
> Section 3.2.12
>
> SubAnnotationPropertyOf(foaf:page :specRef)?
>
> Section 3.2.13
>
> SubAnnotationPropertyOf(foaf:page :issue)?
>
> Section 3.2.14
>
> Annotation(rdfs:isDefinedBy <url to the conformance and test  
> specification>)?
>
> wording
> "# The following "intersection properties" have not been described in
> the test and conformance document but are used"
>
> s/described/enumerated/
>
> They are described in 3.2.1
>
> Section 3.5.1
>
> "instead of using multiple files for each involved ontology"
> s/multiple/separate/
>

Received on Thursday, 2 April 2009 22:33:05 UTC