Re: comments on 28 April Working Draft of EARL Schema

Dear Diego,

Thank you for your positive feedback. You may be interested to learn 
that we have revised this entire conformance section in our latest 
internal draft:
  - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090629#conformance>

We have not addressed the issue of *should* statements in more detail 
than in the previous iteration but we are looking for ways to tighten 
this section up. Please feel free to send further thoughts during the 
upcoming review period for the Last Call publication.

As to the class duality issue, we still see a strong need for providing 
built-in values as well as categories of values. In fact, we think that 
this would increase the consistency among reports from different types 
of applications. Again, we would be happy to hear suggestions about how 
to accomplish this use case effectively during the upcoming LC review.

Best wishes,
   Shadi


Diego Berrueta Muñoz wrote:
> Dear Shadi,
> 
> Thank you again for addressing our comments. We can live with the new 
> draft, although we don't agree with some of your decisions.
> 
> Particularly, the issue of class-instance duality has been solved only 
> partially. We welcome the changes in the textual description of the 
> subclasses, but we still see some redundancy and possibility of 
> inconsistent reports (see our points 2 and 3 in our original message).
> 
> Finally, we would like to remark that we are still uneasy wrt the 
> "should" conformance requirements. Even with new sentence, we think that 
> an objective validation of a report is still impossible.
> 
> Anyway, we don't want to further delay EARL, so it's enough for us to 
> express our concerns, and please continue with the publication of the 
> specification.
> 
> Best,
> 
> El 18/06/2009, a las 12:48, Shadi Abou-Zahra escribió:
> 
>> Dear Diego,
>>
>> Thank you for your comments on the 28 April Working Draft of the 
>> Evaluation and Report Language (EARL) 1.0 Schema publication.
>>
>> ERT WG has considered each of your comments. Please find and updated 
>> Editor's Draft and RDF file incorporating your comments:
>> - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>
>> - <http://www.w3.org/WAI/ER/EARL10/earl.rdf>
>>
>> Find below some background on how they were addressed, please let us 
>> know if you have further thoughts:
>>
>>
>> Diego Berrueta wrote:
>>> Dear participants of the ERT WG,
>>> We are pleased to submit some comments to the 28 April draft of the EARL
>>> schema specification. Firstly, a short self-introduction is in order to
>>> avoid confusions. Luis Polo and I work for CTIC, as Carlos Iglesias
>>> does, but in a different department. CarlosI regularly pings us about
>>> the updates on EARL. Moreover, we are the developers of Vapour [1], an
>>> online validation tool which is available at [2]. Vapour internally uses
>>> EARL (and HTTP-in-RDF) to capture the results of its analysis.
>>> Obviously, it also exports the results in EARL (look for the "RDF" link
>>> at the bottom of any Vapour report).
>>
>> Thank you, we hope that Vapour could be one of the implementations for 
>> the Candidate Recommendation phase.
>>
>>
>>> We think EARL is a flexible and very useful piece of work, and we are
>>> willing to see it published as a REC as soon as possible. However, we
>>> think a few changes can significatively improve the specification.
>>> Please read our comments below, and feel free to engage in discussion
>>> about any of them.
>>> Best regards,
>>> [1] http://vapour.sourceforge.net/
>>> [2] http://validator.linkeddata.org/
>>> =======
>>> FOAF
>>> ****
>>>
>>>> From a technical point of view, we are not completely sure whether the
>>> semantics of foaf:Document are a perfect fit in the context of EARL.
>>> Apparently foaf:Document is designed for human-oriented documents, while
>>> validation usually applies to any kind of document (i.e., we believe
>>> that the semantics of foaf:Document is too narrow). Our suggestion is to
>>> contact FOAF editors to clarify the intended meaning of foaf:Document,
>>> to see, for instance, if it is intended to cover the XML representation
>>> of a human-oriented document, which can be two different subjects of
>>> validation.
>>
>> The FOAF specification currently reads:
>> - "We do not (currently) distinguish precisely between physical and 
>> electronic documents, or between copies of a work and the abstraction 
>> those copies embody."
>>
>> From this, we do not conclude that foaf:Document can not be used to 
>> represent electronic files. We have updated the wording in the current 
>> Editor's Draft and have kept the editor's note to gather further input:
>> - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#TestSubject>
>>
>>
>>> DUBLIN CORE
>>> ***********
>>> The use of the old dc: namespace is not advised in new applications. We
>>> suggest to use exclusively the dct: namespace, which replaces and
>>> extends the obsolete dc: namespace (see the last paragraph of
>>> http://dublincore.org/documents/dces/ : "Over time, however,
>>> implementers are encouraged to use the semantically more precise
>>> dcterms: properties")
>>
>> Thank you, this has been fixed throughout the current Editor's Draft.
>> - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>
>>
>>
>>> CLASS-INSTANCE DUALITY
>>> **********************
>>> Section 2.7 defines 5 subclasses of earl:OutcomeValue, and a prototype
>>> instance of each of these subclasses. We have some comments regarding
>>> these subclasses and instances:
>>> 1) The textual definitions of the instances (e.g., earl:passed) and the
>>> subclasses (e.g., earl:Pass) are identical. Therefore, it follows that
>>> they share the very same conceptual space. Consequently, these singleton
>>> instances are also the only ones possible of their kind, i.e., it is
>>> impossible to conceive an instance of earl:Pass different from
>>> earl:passed.
>>> 2) The duality introduces unnecessary confusion. For instance, under
>>> which condition two tests have the same outcome value? Intuitively, they
>>> have the same outcome if they have the same value (resource) for their
>>> respective earl:outcome properties (e.g., earl:failed, earl:passed...).
>>> However, under the duality, the values cannot be directly compared, as
>>> they could be two different instances of the same subclass (earl:Fail,
>>> earl:Pass...). This greatly obfuscates any SPARQL query on the report.
>>> 3) The duality introduces confusion on the use of meta-modelling. For
>>> instance, which one of the following is correct?
>>>    a) ex:myTestResult1 earl:outcome earl:passed .
>>>   b) ex:myTestResult2 earl:outcome earl:Pass .
>>>    In RDF, there is enough syntactic freedom to use any of these
>>> statements. Unfortunately, this freedom makes it more difficult to
>>> compare the results of two tests, because they do not have the same
>>> ontological status. Moreover, the latter option turns into a problem if
>>> the report is to be interpreted as OWL-DL, because it introduces
>>> meta-modelling.
>>> Consequently, our suggestion is to drop the five subclasses.
>>
>> These five subclasses correspond to an important use-case, namely to 
>> allow the creation of arbitrary instances yet preserve a basic set of 
>> categories for their meaning.
>>
>> We have updated the names, labels, and definitions for these terms to 
>> better clarify their meanings. We've also clarified the use-case in 
>> the editor's note to gather further input:
>> - 
>> <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#OutcomeValue>
>>
>>
>>> CONFORMING REPORTS
>>> ******************
>>> We have two comments with respect to the Conforming Reports (Section
>>> 4.1):
>>> 1) From our point of view, it is not clear enough how "SHOULD"
>>> requirements are to be interpreted. Which are the precise conditions
>>> under which a report passes or fails a SHOULD requirement? It seems to
>>> us that these kind of requirements have little use in practice, and
>>> consequently, we suggest to drop them.
>>> 2) It is not clear whether the conformance requirements must take into
>>> account the RDF(S) semantics. For instance, is the following report
>>> valid wrt requirement 2?
>>>     ex:assertion rdf:type ex:MyAssertionClass .
>>>    ex:MyAssertionClass rdfs:subClassOf earl:Assertion .
>>> Strictly speaking, there isn't any earl:Assertion in the report.
>>> However, RDF(S) Semantics dictate that the triples above entail
>>> (ex:assertion, rdf:type, earl:Assertion), thus making the requirement
>>> pass.
>>> We suggest to explicitly clarify whether the RDF(S) entailment rules
>>> must be applied before checking the conformance rules.
>>
>> We expect that they must be applied, and have updated the descriptions 
>> accordingly:
>> - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610#reports>
>>
>>
>>> RDF REPRESENTATION OF EARL
>>> **************************
>>> In the RDF description of the EARL ontology, please consider using OWL
>>> constructors such as owl:Class. Note that it is not our suggestion to
>>> drop the RDF/RDFS constructors (such as rdfs:Class), which should be
>>> preserved. Our suggestion is to add the OWL constructors in addition to
>>> the current ones. Using both sets of constructors is harmless and cheap
>>> (it just adds a few triples), but enables both the OWL-aware and the
>>> RDFS-aware applications (graphical ontology editors, reasoners, etc.) to
>>> seamlessly operate with the EARL ontology.
>>
>> We have updated the RDF, your input on this would be valuable:
>> - <http://www.w3.org/WAI/ER/EARL10/earl.rdf>
>>
>>
>>> Additionally, please consider adding a direct link to the RDF
>>> description of the EARL ontology from the HTML document. See for
>>> instance the "(rdf)" links at the top of the FOAF HTML document at
>>> http://xmlns.com/foaf/spec/ .
>>
>> We've incorporated this suggestion, thank you for raising it:
>> - <http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20090610>
>>
>>
>> Regards,
>>  Shadi
>>
>> -- 
>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
>>  WAI International Program Office Activity Lead   |
>> W3C Evaluation & Repair Tools Working Group Chair |
> 
> -- 
> Diego Berrueta Muñoz
> Departamento de I+D+I  -  Fundación CTIC
> -Centro Tecnológico de la Información y la Comunicación-
> E-mail: diego.berrueta@fundacionctic.org
> Tfno:+34 984 29 12 12
> Parque Científico Tecnológico Gijón-Asturias-Spain
> www.fundacionctic.org
> 
> 

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
   WAI International Program Office Activity Lead   |
  W3C Evaluation & Repair Tools Working Group Chair |

Received on Monday, 6 July 2009 12:29:44 UTC