W3C home > Mailing lists > Public > public-wai-ert@w3.org > June 2007

RE: EARL vs. our desired test results format

From: Jo Rabin <jrabin@mtld.mobi>
Date: Tue, 19 Jun 2007 15:28:31 +0000
Message-ID: <C8FFD98530207F40BD8D2CAD608B50B43BC236@mtldsvr01.DotMobi.local>
To: <public-mobileok-checker@w3.org>, <public-wai-ert@w3.org>

As a matter of principle, I think it is desirable for us to stick with
W3C formats and extend where necessary. However, I also think there are
practical obstacles. The principal obstacle I see is that none of us on
this team are au fait with RDF. In the course of development of this
result format I imagine that there will be a fair amount of detailed
to-ing and fro-ing. The fact that we are collectively going to struggle
with even basic things like sub-classing is not really a good start.

In line with Roland's suggestion, I think it will be altogether more
practical for us to work in our own lingua franca and look at turning
this into EARL later, no doubt with a little help from our friends.


> -----Original Message-----
> From: public-mobileok-checker-request@w3.org [mailto:public-mobileok-
> checker-request@w3.org] On Behalf Of Shadi Abou-Zahra
> Sent: 19 June 2007 11:43
> To: Sean Owen
> Cc: public-mobileok-checker@w3.org; public-wai-ert@w3.org
> Subject: Re: EARL vs. our desired test results format
> Hi Sean, checker folks,
> It's understandable if you can not wait for EARL and want to cook your
> own soup. However, I think there are some misunderstandings about EARL
> and I'd like to clarify some below. Also, I think it may be worth
> to consider extending EARL where it doesn't meet your specific goals
> rather than reinvent everything from scratch. Some of these additions
> may eventually make into EARL at some point too.
> Sean Owen wrote:
> > 1. EARL does not have a notion of test suites as far as I can tell
> > groups of tests. We have three levels here really -- the overall
> > of a resource's mobileOK-ness, comprised of many mobileOK test
> > outcomes, each comprised of several warns and failures. Right now we
> > sort of finesse this by pretending the overall test isn't a test,
> > now that we have a third level this is problematic.
> This is the beauty of RDF -you can extend EARL with a custom test case
> description vocabulary. EARL provides a TestCriterion stub with two
> "built-in" sub-classes TestRequirement and TestCase. We think that
> terms, while not closely defined, can be further extended to serve
> different situations including mobileOK. It would be useful feedback
> you can let us know why these terms can not be extended to serve your
> specific needs.
> > 2. No suitable notion of "position" and code snippet. There's a
> > of a pointer to the source doc but it's an XPath  pointer as far as
> > can tell.
> Actually EARL provides many different types of pointers, including
> (and byte) snippets. Sadly this work is currently going a little
> than expected but you can see first attempts in the "Pointer Methods
> RDF" document:
>   - <http://www.w3.org/WAI/ER/Pointers/WD-Pointers-20070222>
> If you can not wait for this pointer vocabulary then you can just
> your own as sub-classes of earl:pointer. You can focus your energy on
> creating the new terms rather than reinventing the whole framework.
> > 3. EARL defines no real XML serialization. Not like we can't make up
> > the obvious one, but, it's a point.
> EARL does not define an XML Schema or DTD, but we use RDF/XML as the
> serialization. We are also currently working on a comment on this
> to provide an XML Schema. There is no group decision on this yet but
> you develop your own XML Schema for EARL we may be inclined to adopt
> > 4. EARL in XML is verbose and hard to understand. Here's an example
> result:
> > [...]
> > This is a more general knock of RDF in XML. Or maybe RDF. Ouch! But
> > really, this is the sort of output we want developers to use. They
> > need to make sense of it. We have to write a schema. Ick.
> Granted. We tried to simplify EARL as much as possible but this is
> clearly the downside of RDF/XML. On the other hand, how many users
> really need to read the XML all the time?
> > 5. EARL has no notion of warnings, though we can conflate it with
> > "info" without too much trouble. Still a point.
> We've had extensive debate about this in the past and decided that a
> warning is usually a subclass of pass. As I understood, also in
> a WARN is not a real result but an informative message before a PASS
> FAIL result. This is why we've added "info", to serve these purposes.
> you really need a warning *result* then just sub-class it as needed.
> > So... I suggest we jump to a simpler, custom-built XML format like
> > one we sketched last week. I don't think it's because EARL is
> > deficient, but because we have pretty detailed test output needs,
> > also because it's RDF-ness just adds complexity here that doesn't
> > -- well, that's the view from my brain, which has never really
> > "gotten" the point of RDF.
> The primary motivation for creating EARL as an RDF format is to give
> the necessary flexibility and extensibility, yet provide a sufficient
> framework to exchange basic information on test results.
> Anyway, as mentioned earlier this is not an attempt to convince you
> to clarify some of the issues. We would be very interested to learn
> about why EARL does not satisfy your specific needs and happier to
> about any extensions you develop.
> Regards,
>    Shadi
> --
> Shadi Abou-Zahra     Web Accessibility Specialist for Europe |
> Chair & Staff Contact for the Evaluation and Repair Tools WG |
> World Wide Web Consortium (W3C)           http://www.w3.org/ |
> Web Accessibility Initiative (WAI),   http://www.w3.org/WAI/ |
> WAI-TIES Project,                http://www.w3.org/WAI/TIES/ |
> Evaluation and Repair Tools WG,    http://www.w3.org/WAI/ER/ |
> 2004, Route des Lucioles - 06560,  Sophia-Antipolis - France |
> Voice: +33(0)4 92 38 50 64          Fax: +33(0)4 92 38 78 22 |
Received on Tuesday, 19 June 2007 21:27:36 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:55:55 UTC