Re: Conformance Evaluation

Crossposted to AU and ER

Len identifies some important points.

1. It is indeed useful to identify what tools meet the requirements. In fact,
W3C process requires that this can be done for a recommendation. So we have
two options - we can build the tools ourselves, or find tools that already
implement the requirements. It is also useful for developers, who are trying
to meet the guidelines, to have more guidance about what that means. Seeing
the opinions of the working group is helpful in this regard. For the working
group to have a good idea of how to make decisions on "line calls", and even
more so for others, it is helpful to have a number of such decisions
available to refer to.

2. There are indeed questions about what does "integrated into the look and
feel mean". Again, the best way I know of to clarify this is by trying
it. Some of the other cases, like the relative priority checkkpoints, are
less complicated. There is a lot of work to test each one, but it is
explained under relative priority that what is required is based on the
checkpoints of WCAG. Developing techniques, and tests, is a lot of the
current work in front of the AU group.

3. Vendor neutral is not the same as issue neutral, or silent. W3C is
certainly not in a position to sit down and work on their favourite vendor's
tools, or say that someone's software is all terrible. However, in order to
demonstrate what we mean by the specification, in informative examples, it is
reasonable to publish opinions expressed about how a particular tool does or
does not meet the requirements, along with any subsequent debate on the
issue. Not to do so would be tantamount to claiming the specification is
perfect, and anyone who doesn't understand it is inadequate, which is likely
to be almost diametrically wrong. Our comments, if we are working in good
faith, are not likely to provide undeserved praise or blame in any great
degree, and as a working group it is our job to watch over ourselves and each
other, and test what we say against reality. I believe we are in fact capable
of doing so (although I recognise that it is important to bear this in mind,
and recall that we are none of us infallible).

On another front, in the ER response to this thread, Kynn asked what was
happening about the conformance and quality assurance person W3C was going to
hire. As I understand it, his name is Karl Dubost and he will be beginning
work at the start of July. (People who follow everything in AU may notice
that he is in fact the person who produced the french translation of ATAG.

Charles McCN

On Fri, 9 Jun 2000, Leonard R. Kasday wrote:

  There was a discussion in the last teleconference about providing a 
  database telling whether various authoring tools meet ATAG.
  
  I'd like to focus on one of the issues.  Do we rely on the tool providers 
  to do self evaluation?  Or does the W3C take on the role of evaluating tools?
  
  Taking off my chair hat I offer a few observations:
  
  1. It would undoubtedly be useful for people to have an accurate account of 
  whether tools meet ATAG.
  
  2. However, some of the evaluation is subjective.  For example, 3.2 
  requires "Help the author create structured content and separate 
  information from its presentation."  How much help is required to say this 
  is satisfied, e.g. how many of the techniques in 
  http://www.w3.org/TR/ATAG10-TECHS/#check-help-provide-structure ?  Just 
  one? Five? etc.   Similarly 4.1, checking for accessibility, references 
  AERT, not a normative document, so evaluating satisfaction is 
  subjective.  Checkpoint 5.2's "obvious and easily initiated" also worries 
  me.  As a human factors engineer, I've had lots of debates with people who 
  thought something was obvious and easily intiated when the users didn't.
  
  3. W3C is supposed to be vendor neutral.
  
  So, what I'm worried about is complaints about our subjective judgments 
  favoring one company or another.
  
  Is there a subset of strictly objective specs that avoids this problem?
  
  OK, chair hat back on.  Any comments?
  
  Len
  
  p.s.
  I copied this to Judy given the policy aspects.
  --
  Leonard R. Kasday, Ph.D.
  Institute on Disabilities/UAP, and
  Department of Electrical Engineering
  Temple University 423 Ritter Annex, Philadelphia, PA 19122
  
  kasday@acm.org
  http://astro.temple.edu/~kasday
  
  (215) 204-2247 (voice)  (800) 750-7428 (TTY)
  
  The WAVE web page accessibility evaluation assistant: 
  http://www.temple.edu/inst_disabilities/piat/wave/
  

--
Charles McCathieNevile    mailto:charles@w3.org    phone: +61 (0) 409 134 136
W3C Web Accessibility Initiative                      http://www.w3.org/WAI
Location: I-cubed, 110 Victoria Street, Carlton VIC 3053
Postal: GPO Box 2476V, Melbourne 3001,  Australia

Received on Friday, 9 June 2000 22:56:02 UTC