RE: EARL vs. our desired test results format

And, besides, the design of the checker (accompanied by the Developer
Guide that is being created in parallel) will make it easy for the
developer community to enhance the checker with EARL output support
instead of our own lingua franca. This might be made by members of this
task force, by ERT WG members interested into the support of EARL in the
checker, or by anyone in the open source community, in a future version
2.0 of this software.

Regards,

Nacho

-----Mensaje original-----
De: public-mobileok-checker-request@w3.org
[mailto:public-mobileok-checker-request@w3.org] En nombre de Jo Rabin
Enviado el: martes, 19 de junio de 2007 17:28
Para: public-mobileok-checker@w3.org; public-wai-ert@w3.org
Asunto: RE: EARL vs. our desired test results format


As a matter of principle, I think it is desirable for us to stick with
W3C formats and extend where necessary. However, I also think there are
practical obstacles. The principal obstacle I see is that none of us on
this team are au fait with RDF. In the course of development of this
result format I imagine that there will be a fair amount of detailed
to-ing and fro-ing. The fact that we are collectively going to struggle
with even basic things like sub-classing is not really a good start.

In line with Roland's suggestion, I think it will be altogether more
practical for us to work in our own lingua franca and look at turning
this into EARL later, no doubt with a little help from our friends.

Jo

> -----Original Message-----
> From: public-mobileok-checker-request@w3.org [mailto:public-mobileok-
> checker-request@w3.org] On Behalf Of Shadi Abou-Zahra
> Sent: 19 June 2007 11:43
> To: Sean Owen
> Cc: public-mobileok-checker@w3.org; public-wai-ert@w3.org
> Subject: Re: EARL vs. our desired test results format
> 
> 
> Hi Sean, checker folks,
> 
> It's understandable if you can not wait for EARL and want to cook your
> own soup. However, I think there are some misunderstandings about EARL
> and I'd like to clarify some below. Also, I think it may be worth
while
> to consider extending EARL where it doesn't meet your specific goals
> rather than reinvent everything from scratch. Some of these additions
> may eventually make into EARL at some point too.
> 
> 
> Sean Owen wrote:
> > 1. EARL does not have a notion of test suites as far as I can tell
--
> > groups of tests. We have three levels here really -- the overall
test
> > of a resource's mobileOK-ness, comprised of many mobileOK test
result
> > outcomes, each comprised of several warns and failures. Right now we
> > sort of finesse this by pretending the overall test isn't a test,
but
> > now that we have a third level this is problematic.
> 
> This is the beauty of RDF -you can extend EARL with a custom test case
> description vocabulary. EARL provides a TestCriterion stub with two
> "built-in" sub-classes TestRequirement and TestCase. We think that
these
> terms, while not closely defined, can be further extended to serve
many
> different situations including mobileOK. It would be useful feedback
if
> you can let us know why these terms can not be extended to serve your
> specific needs.
> 
> 
> > 2. No suitable notion of "position" and code snippet. There's a
notion
> > of a pointer to the source doc but it's an XPath  pointer as far as
I
> > can tell.
> 
> Actually EARL provides many different types of pointers, including
code
> (and byte) snippets. Sadly this work is currently going a little
slower
> than expected but you can see first attempts in the "Pointer Methods
in
> RDF" document:
>   - <http://www.w3.org/WAI/ER/Pointers/WD-Pointers-20070222>
> 
> If you can not wait for this pointer vocabulary then you can just
create
> your own as sub-classes of earl:pointer. You can focus your energy on
> creating the new terms rather than reinventing the whole framework.
> 
> 
> > 3. EARL defines no real XML serialization. Not like we can't make up
> > the obvious one, but, it's a point.
> 
> EARL does not define an XML Schema or DTD, but we use RDF/XML as the
XML
> serialization. We are also currently working on a comment on this
draft
> to provide an XML Schema. There is no group decision on this yet but
if
> you develop your own XML Schema for EARL we may be inclined to adopt
it.
> 
> 
> > 4. EARL in XML is verbose and hard to understand. Here's an example
> result:
> > [...]
> > This is a more general knock of RDF in XML. Or maybe RDF. Ouch! But
> > really, this is the sort of output we want developers to use. They
> > need to make sense of it. We have to write a schema. Ick.
> 
> Granted. We tried to simplify EARL as much as possible but this is
> clearly the downside of RDF/XML. On the other hand, how many users
> really need to read the XML all the time?
> 
> 
> > 5. EARL has no notion of warnings, though we can conflate it with
> > "info" without too much trouble. Still a point.
> 
> We've had extensive debate about this in the past and decided that a
> warning is usually a subclass of pass. As I understood, also in
mobileOK
> a WARN is not a real result but an informative message before a PASS
or
> FAIL result. This is why we've added "info", to serve these purposes.
If
> you really need a warning *result* then just sub-class it as needed.
> 
> 
> > So... I suggest we jump to a simpler, custom-built XML format like
the
> > one we sketched last week. I don't think it's because EARL is
> > deficient, but because we have pretty detailed test output needs,
and
> > also because it's RDF-ness just adds complexity here that doesn't
help
> > -- well, that's the view from my brain, which has never really
> > "gotten" the point of RDF.
> 
> The primary motivation for creating EARL as an RDF format is to give
it
> the necessary flexibility and extensibility, yet provide a sufficient
> framework to exchange basic information on test results.
> 
> Anyway, as mentioned earlier this is not an attempt to convince you
but
> to clarify some of the issues. We would be very interested to learn
more
> about why EARL does not satisfy your specific needs and happier to
learn
> about any extensions you develop.
> 
> Regards,
>    Shadi
> 
> 
> --
> Shadi Abou-Zahra     Web Accessibility Specialist for Europe |
> Chair & Staff Contact for the Evaluation and Repair Tools WG |
> World Wide Web Consortium (W3C)           http://www.w3.org/ |
> Web Accessibility Initiative (WAI),   http://www.w3.org/WAI/ |
> WAI-TIES Project,                http://www.w3.org/WAI/TIES/ |
> Evaluation and Repair Tools WG,    http://www.w3.org/WAI/ER/ |
> 2004, Route des Lucioles - 06560,  Sophia-Antipolis - France |
> Voice: +33(0)4 92 38 50 64          Fax: +33(0)4 92 38 78 22 |

Received on Wednesday, 20 June 2007 11:42:19 UTC