Re: comments on 28 April Working Draft of EARL Schema

Dear Diego,

Thank you for your comments on the 28 April Working Draft of the 
Evaluation and Report Language (EARL) 1.0 Schema publication.

ERT WG has considered each of your comments. Please find and updated 
Editor's Draft and RDF file incorporating your comments:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>
  - <http://www.w3.org/WAI/ER/EARL10/earl.rdf>

Find below some background on how they were addressed, please let us 
know if you have further thoughts:


Diego Berrueta wrote:
> Dear participants of the ERT WG,
> 
> We are pleased to submit some comments to the 28 April draft of the EARL
> schema specification. Firstly, a short self-introduction is in order to
> avoid confusions. Luis Polo and I work for CTIC, as Carlos Iglesias
> does, but in a different department. CarlosI regularly pings us about
> the updates on EARL. Moreover, we are the developers of Vapour [1], an
> online validation tool which is available at [2]. Vapour internally uses
> EARL (and HTTP-in-RDF) to capture the results of its analysis.
> Obviously, it also exports the results in EARL (look for the "RDF" link
> at the bottom of any Vapour report).

Thank you, we hope that Vapour could be one of the implementations for 
the Candidate Recommendation phase.


> We think EARL is a flexible and very useful piece of work, and we are
> willing to see it published as a REC as soon as possible. However, we
> think a few changes can significatively improve the specification.
> Please read our comments below, and feel free to engage in discussion
> about any of them.
> 
> Best regards,
> 
> [1] http://vapour.sourceforge.net/
> [2] http://validator.linkeddata.org/
> 
> =======
> 
> FOAF
> ****
>  
>>From a technical point of view, we are not completely sure whether the
> semantics of foaf:Document are a perfect fit in the context of EARL.
> Apparently foaf:Document is designed for human-oriented documents, while
> validation usually applies to any kind of document (i.e., we believe
> that the semantics of foaf:Document is too narrow). Our suggestion is to
> contact FOAF editors to clarify the intended meaning of foaf:Document,
> to see, for instance, if it is intended to cover the XML representation
> of a human-oriented document, which can be two different subjects of
> validation.

The FOAF specification currently reads:
  - "We do not (currently) distinguish precisely between physical and 
electronic documents, or between copies of a work and the abstraction 
those copies embody."

 From this, we do not conclude that foaf:Document can not be used to 
represent electronic files. We have updated the wording in the current 
Editor's Draft and have kept the editor's note to gather further input:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#TestSubject>


> DUBLIN CORE
> ***********
>  
> The use of the old dc: namespace is not advised in new applications. We
> suggest to use exclusively the dct: namespace, which replaces and
> extends the obsolete dc: namespace (see the last paragraph of
> http://dublincore.org/documents/dces/ : "Over time, however,
> implementers are encouraged to use the semantically more precise
> dcterms: properties")

Thank you, this has been fixed throughout the current Editor's Draft.
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>


> CLASS-INSTANCE DUALITY
> **********************
>  
> Section 2.7 defines 5 subclasses of earl:OutcomeValue, and a prototype
> instance of each of these subclasses. We have some comments regarding
> these subclasses and instances:
>  
> 1) The textual definitions of the instances (e.g., earl:passed) and the
> subclasses (e.g., earl:Pass) are identical. Therefore, it follows that
> they share the very same conceptual space. Consequently, these singleton
> instances are also the only ones possible of their kind, i.e., it is
> impossible to conceive an instance of earl:Pass different from
> earl:passed.
>  
> 2) The duality introduces unnecessary confusion. For instance, under
> which condition two tests have the same outcome value? Intuitively, they
> have the same outcome if they have the same value (resource) for their
> respective earl:outcome properties (e.g., earl:failed, earl:passed...).
> However, under the duality, the values cannot be directly compared, as
> they could be two different instances of the same subclass (earl:Fail,
> earl:Pass...). This greatly obfuscates any SPARQL query on the report.
>  
> 3) The duality introduces confusion on the use of meta-modelling. For
> instance, which one of the following is correct?
>  
>    a) ex:myTestResult1 earl:outcome earl:passed .
>    b) ex:myTestResult2 earl:outcome earl:Pass .
>  
>    In RDF, there is enough syntactic freedom to use any of these
> statements. Unfortunately, this freedom makes it more difficult to
> compare the results of two tests, because they do not have the same
> ontological status. Moreover, the latter option turns into a problem if
> the report is to be interpreted as OWL-DL, because it introduces
> meta-modelling.
>  
> Consequently, our suggestion is to drop the five subclasses.

These five subclasses correspond to an important use-case, namely to 
allow the creation of arbitrary instances yet preserve a basic set of 
categories for their meaning.

We have updated the names, labels, and definitions for these terms to 
better clarify their meanings. We've also clarified the use-case in the 
editor's note to gather further input:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#OutcomeValue>


> CONFORMING REPORTS
> ******************
>  
> We have two comments with respect to the Conforming Reports (Section
> 4.1):
>  
> 1) From our point of view, it is not clear enough how "SHOULD"
> requirements are to be interpreted. Which are the precise conditions
> under which a report passes or fails a SHOULD requirement? It seems to
> us that these kind of requirements have little use in practice, and
> consequently, we suggest to drop them.
>  
> 2) It is not clear whether the conformance requirements must take into
> account the RDF(S) semantics. For instance, is the following report
> valid wrt requirement 2?
>  
>     ex:assertion rdf:type ex:MyAssertionClass .
>     ex:MyAssertionClass rdfs:subClassOf earl:Assertion .
>  
> Strictly speaking, there isn't any earl:Assertion in the report.
> However, RDF(S) Semantics dictate that the triples above entail
> (ex:assertion, rdf:type, earl:Assertion), thus making the requirement
> pass.
>  
> We suggest to explicitly clarify whether the RDF(S) entailment rules
> must be applied before checking the conformance rules.

We expect that they must be applied, and have updated the descriptions 
accordingly:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#reports>


> RDF REPRESENTATION OF EARL
> **************************
>  
> In the RDF description of the EARL ontology, please consider using OWL
> constructors such as owl:Class. Note that it is not our suggestion to
> drop the RDF/RDFS constructors (such as rdfs:Class), which should be
> preserved. Our suggestion is to add the OWL constructors in addition to
> the current ones. Using both sets of constructors is harmless and cheap
> (it just adds a few triples), but enables both the OWL-aware and the
> RDFS-aware applications (graphical ontology editors, reasoners, etc.) to
> seamlessly operate with the EARL ontology.

We have updated the RDF, your input on this would be valuable:
  - <http://www.w3.org/WAI/ER/EARL10/earl.rdf>


> Additionally, please consider adding a direct link to the RDF
> description of the EARL ontology from the HTML document. See for
> instance the "(rdf)" links at the top of the FOAF HTML document at
> http://xmlns.com/foaf/spec/ .

We've incorporated this suggestion, thank you for raising it:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>


Regards,
   Shadi

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
   WAI International Program Office Activity Lead   |
  W3C Evaluation & Repair Tools Working Group Chair |

Received on Thursday, 18 June 2009 10:49:33 UTC