Re: What Is EARL?

At 08:21 PM 2001-07-09 , Sean B. Palmer wrote:
>Since more and more people are starting to ask, we should probably
>have some definitive answer that we all agree on (for any value of
>all).
>
>So, here's my first attempt at an elevator hook, arranged as "what",
>"why", and "how":-
>
>What:-
>EARL - the Evaluation And Report Language - is an RDF based
>system/model/framework/+ for making scoped evaluations about
>resources/things/RDF resources/+, being developed by the W3C WAI
>Evaluation and Repair Tools group. EARL is non constraining in the
>range of things that can be evaluated, or the range of conformance
>criteria that these things can be evaluated against, but it does
>provide a vocabulary to facilitate scoped reports.
>
>Why:-
>EARL can be used for a number of applications, for example, Quality
>Assurance, simple Web site ratings (er... too much like PICS in some
>senses), authoring tool and user agent ratings and bug reports, device
>independence testing and rating, and so on. Being based on RDF, EARL
>is also Semantic Web compatable, and tools produced by the W3C's
>Semantic Web activity may be able to be used on EARL.
>
>How:-
>In general, an EARL evaluation consists of a context, and then an
>assertion which consists of the thing being evaluated, the conformance
>criteria, and the validity status. For example, contextual information
>may include information such as creator details, platform, and so on.
>The thing beiong evaluated could be a Web page, or a tool. The
>conformance criteria could be something like a WCAG checkpoint or a
>syntax rule in a schema, and the validity status could be something as
>simple as "pass or fail" or something more granular with, for example,
>a certain level of confidence.
>

Here's another run at that triple.

What is EARL?  A notation for recording and sharing evaluations.

Evaluation ::= tuple (
  what was evaluated -- any referenceable scope of resource
  against what criteria -- published or re-usable assesment instrument as
applicable
  with what conclusion -- outcome, consistent with the conventions of
instrument
  under what other conditions -- computing environment, human exercising
judgement, ad lib.
  )

Why EARL?  The people who can tell you what works and what doesn't work are
often different people, operating on different computers, from the people who
can discern why it did or didn't work in those cases, and how to repair
defects
and extend positive results into wider applicability.

Product developers need to accumulate evaluations from many independent
evaluators.  Policy monitors need to collect comparable evaluations for many
evaluands.  Free and machine-comprehensible exchange of this information will
allow information collections to reach critical mass and clearly point the way
for action.

Who:  EARL is an experiment of the W3C/WAI Evaluation and Repair Tools Working
Group <<http://www.w3.org/WAI/ER/>http://www.w3.org/WAI/ER/>.

Why WAI-ER?  The need for something that can do what EARL does is especially
important if one wishes to improve information access by people with
disabilities.  Existing practices of user testing become prohibitively
difficult for this user group.  It takes Internet-based methods to reduce user
evaluation to an affordable cost in this community.  People with disabilities
commonly experience mobility problems which makes it harder to get them to a
controlled, dedicated evaluation laboratory.  Likewise, there is a vast
diversity of equipment configurations that people use to accomodate their
needs.  This makes it difficult to build a large enough sample of
respondents. 
The laboratory would have to be perpetually reconfigured to provide a
realistic
emulation of their environment of use.  All of this argues in favor of tooling
up for in_vivo testing -- going straight to the field, or at least into
laboratories where the primary activity is fitting and training assitive
technology.  Here the evaluations are performed in the user's operational
environement where the assitive features have been installed and configured
and
work.  Some means is needed, for these reasons, to a) isolate the information
that is usefully comparable across user experiences, and b) automate its
exchange so that small (down to one user at home) laboratories can effectively
contribute usable information into collections.

How does EARL work?  This notation is being built as an RDF application,
composed of a small starter vocabulary of RDF terms which, by RDF semantics,
create pattern definitions.  These patterns provide just enough structure so
you can put together records of evaluations suitable for machine exchange and
processing, but can say whatever it is that you have to say.  Employing the
RDF
framework as an implementation medium means that the groundwork has already
been laid so one can freely extend the vocabulary while preserving the
processability of the shared core concepts.  It also brings access to a
growing
base of tools that process RDF, including products from the W3C Semantic Web
Initiative.

[end draft]

Notes:

When I stated the "why WAI-ER" story -- that we need to try harder on how to
gather user experience among the PWD user population -- at the "Accessibility
Forum" meeting in the U.S., industry people stopped arguing and started taking
notes.  I think that there may be more latent support for EARL out there than
we realize.

I am talking through my hat in the extensibility claims in the last
paragraph. 
This could be toned down, or an explanation of why this is true developed
and/or cited.

But I would strongly urge that the "what" not say RDF, but what information is
to be captured and exchanged.  RDF is a 'how,' please.

Al

>--
>Kindest Regards,
>Sean B. Palmer
>@prefix : <<http://webns.net/roughterms/>http://webns.net/roughterms/> .
>:Sean :hasHomepage <<http://purl.org/net/sbp/>http://purl.org/net/sbp/> .
>

Received on Tuesday, 10 July 2001 09:52:01 UTC