Re: ERT WG proposal for discussion at F2F

My main thoughts after reading Wendy's summary are that we have the 
following interactions with AU and GL

Write GL so that they are machine readable. Then an evaluation tool could 
pull in GL and write EARL.
For example, if GL were in form

Element X must/should/may have attribute Y, which a human judges according 
to the following criteria
[pointer to explanation of criteria which is only human readable]

(This is like a schema with additional pointers to human-readable criteria)

Then a tool would have a general function that checks for the existance of 
the attribute, presents the human readable criteria,  and gives the human 
user a scale to on which to set a value.

The tool then exports result as EARL.

An editing tool would accept this info and integrate it into it's operation.

With this general scheme you wouldn't need separate hand written code for 
all the different attributes that fit this scheme, like ALT, SUMMARY, 
TITLE, etc.  They would all be treated the same.

Additional thoughts below.  See esp.

I labeled my notes "LRK::" and Wendy's with "WC::"
There also references to Seans followup that he wrote under the title "EARL 
Overview"

At 03:03 PM 2/15/01 -0500, Wendy A Chisholm wrote:
WC::
>...
>The ERT WG has been working on an Evaluation and Repair Language 
>(EARL).  The goals of this language are twofold:
>1. to be generated by authoring, evaluation, and repair tools to track the 
>accessibility of a page or a site.
>2. to be shared between authoring, evaluation, and repair tools so as not 
>to replicate work or to build on each other's work.

LRK:: This says how it's generated and that it's shared, but we also need 
to say what it is.

Here's the summary statement from Sean's overview:

"The Evaluation And Repair Language is an RDF based framework for 
recording, transferring and processing data about automatic and manual 
evaluations of resources. The purpose is to provide a framework for generic 
evaluation description formats that can be used in generic evaluation and 
repair tools. "


Here's what I wrote in the scenarios page [3].  It's a special case of 
Sean's general statement, and perhaps we can just append it

(Sean... would you add a pointer to the scenarios from the overview page?)

EARL would initially be a means for expressing in a machine readable form ...
·       results of evaluating web pages and web sites against the Web 
Accessibility Content Guidelines
·       specifications of repairs to those pages @@new proposal: no 
consensus yet.

The language would be extensible to
·       evaluations of languages, e.g. as defined by their schemas
·       evalutions of user agents
·       evaluations of authoring tools
·       evalutions against other types of specs, e.g. validity.
·       @@repair of above??

The Scenarios page then describes the following (see [3] for more details)
Evaluation and Repair Reports
Web Page and Site Repair (using editing tools that accept earl)
Third Party Accessibility Services
Group Judgments of Web Pages

>WC:: Other applications:
>1. To search for sites that conform to WCAG or that do not have certain 
>accessibility issues.
>2. To make conformance claims on any set of guidelines (i.e. could use for 
>ATAG or UAAG conformance claims as well).
>
>More information is available at [1] (thanks to Sean Palmer)


LRK:: I'd second Sean's suggestion to attribute this to the whole ER group

>WC:: At Monday's telecon [2], we discussed various claims that someone 
>might make about an attribute or an element on a site or page.  For 
>example, someone or some tool might say, "image x has an alt 
>attribute.  the contents of the alt attribute are acceptable."
>Or "image x is missing an alt attribute."
>or "image x does not have a longdesc attribute and needs one"
>or "image x does not have a longdesc attribute and does not need one."
>or "the alt-text for image x is ok, but I suggest that you use this text 
>instead."
>Or, "the alt-text for image x  is ok, but could be better."
>
>This lead to a discussion of rating scales.  For example, one complaint 
>that I have heard from people implementing WCAG 1.0 is: I have followed 
>all of the priority 1 checkpoints and all but 1 of the priority 2 
>checkpoints yet I can only claim Level A conformance.  In other words, it 
>is all or nothing.
>
>Perhaps in WCAG 2.0, using EARL someone could claim they have met Level A 
>as well as checkpoints x, y, z.
>
>On an individual checkpoint level, there are many subjective checkpoints 
>in WCAG.  These require a human "rating" of sorts.  A rating implies that 
>a scale is being used.  The scales might be:
>verbosity ("the longdesc needs more detail"),
>appropriateness ("this alt-text is not equivalent"),
>simplicity ("the language is too difficult"),
>etc.

LRK:: You're right that those are relevant axes along which to evaluate, 
but for now I'd be inclined to just have a single variable, e.g. "quality" 
with a free form comment field that the person could use to explain how 
they got there.  Although WCAG could provide these sorts of guidance.

Hmmm.... so WCAG could be written in a machine readable form so that a tool 
could present the WCAG criteria when the user is performing evaluations 
that will be recorded in EARL...

>WC:: Perhaps we should associate a scale and a test procedure that someone 
>can follow to help them determine if they have met each technique/checkpoint?

LRK:: If the test procedure has steps, I'd suggest that EARL actually 
record the result of each step, instead of just the bottom line.

>WC:: (Aside: I still find it very hard to distinguish between checkpoints 
>and techniques - i still think these need to be called like 
>technology-specific checks or something)
>
>Previously, people felt that we could use a point system for each 
>checkpoint. People would add up their points to determine their level of 
>accessibility.  This was discounted with the argument that someone could 
>have a high point total but not be accessible.
>
>What the ERT WG is asking is "can we only answer yes or no or not 
>applicable for a checkpoint?"
>
>The ERT WG proposes that we discuss conformance issues at the joint 
>meeting between AU WG, WCAG WG, and ERT WG during the afternoon of March 1 
>rather than AERT open issues.
>
>If others from the AERT WG would like to help me clarify our questions, I 
>would appreciate it.  I know we have talked some about ratings and decided 
>not to do that for WCAG, but we are not talking about something a bit 
>different this time.


I also updated the home page to point to Sean's new overview page, per his 
followup to this message (titled EARL Overview, thanks Sean)


[1] http://www.w3.org/WAI/ER/IG/#earl
[2] http://www.w3.org/WAI/ER/2001/02/12-minutes.html

[3] http://www.w3.org/WAI/ER/IG/earl.html


--
Leonard R. Kasday, Ph.D.
Institute on Disabilities/UAP and Dept. of Electrical Engineering at Temple 
University
(215) 204-2247 (voice)                 (800) 750-7428 (TTY)
http://astro.temple.edu/~kasday         mailto:kasday@acm.org

Chair, W3C Web Accessibility Initiative Evaluation and Repair Tools Group
http://www.w3.org/WAI/ER/IG/

The WAVE web page accessibility evaluation assistant: 
http://www.temple.edu/inst_disabilities/piat/wave/