Re: Fwd: Accessibility tools

Hi Phil,

Thank you for this input -- very useful thoughts.

Indeed, the WAI Evaluation and Repair Tools Working Group (ERT WG) has 
looked at several approaches in the past but it turned out to be not 
that trivial. For example, what does it mean when we say evaluation tool 
X addresses Success Criteria Y? Some tools will check the code to 
identify whether a particular Success Criteria is applicable, others 
might then do partial checks on the corresponding code (with varying 
degrees of conclusiveness), and yet others might just automatically list 
all Success Criteria "for manual verification" by the evaluator. It 
would be hard to draw a line and say which tool addresses specific 
Success Criteria without a more comprehensive set of test cases. Same 
problem arises for web technology coverage and other checking aspects.

The "WCAG 2.0 Test Samples" is intended to be such a test suite:
  - http://www.w3.org/WAI/ER/tests/

(Much of that work was contributed by the EC-funded BenToWeb Project.)

In parallel, the Auto-WCAG community group is also doing related work on 
defining automated "checks" for WCAG 2.0 rather than test cases:
  - http://www.w3.org/community/groups/#auto-wcag

(That work was initiated by another EC-funded project called EIII.)

Another complimentary approach that we are taking is to describe the 
"features" that different tools could provide, which provides a way of 
better describing what particular tools can and cannot do:
  - http://www.w3.org/TR/WAET/

(This was developed with support from the EC-funded WAI-ACT project.)

So, we have several pieces that we (the community) could build on to 
develop that matrix that you are talking about. It would require quite 
some effort though. I would be delighted to explore opportunities.

Best,
   Shadi


On 9.12.2014 19:25, Phill Jenkins wrote:
> My recommendation:
> I recommend a center of competency (e.g. W3C WAI Evaluation & Repair
> Working Group) maintain a matrix like table of contents or
> cross-referenced searchable index to document the current  and future
> status for which accessibility test tools on which platforms for each WCAG
> 2.0 Success Criteria.   There are at least two approaches to display this
> reference - By WCAG 2.0 Success Criteria: this is what  tools are
> available for each Success Criteria by platform and by browser; versus the
> by platform view: this is the set of platforms and browsers an enterprise
> may have already established, what is the status of accessibility tools?
>
> A screen reader is just one tool set but covers a major set of the 38
> Level A and Double AA Success Criteria.  The automated checker tool(s) is
> a second set of tools.  Color Contrast Analyzers and other browser toolbar
> plug-ins are a third set of tools and often only cover a single or smaller
> set of Success Criteria.
>
> Context: Often we accessibility practitioners get engaged about the
> question: "what is the standard set of tools we should use to validate
> accessibility?"  and we get engaged to deliver courses that include
> training on tools for designers, developers, and testers.  How can the
> enterprise be enables?  The budget for tools and training is often a
> strong consideration in establishing the "standard tool set".  The browser
> and platform standards at an organization is another strong consideration
> for establishing the standard tool set, e.g. we hear a government agency
> say: "but we have to test on IE 9, our standard desktop browser", or the
> bank will say, "it has to run on Firefox ESR", etc.  The other strong
> consideration is the chosen standard set of platforms; iOS 8, Android 5.0,
> Windows 7, ChromeOS 40, etc.
>
> Note: I am NOT talking about the accessibility capital S Standard - based
> on WCAG 2.0, but I'm talking about  the lower case standard set of
> browser, platforms and tools that an enterprise or project standardizes on
> . . .
>
> The WCAG working group alluded to this as the "accessibility supported"
> discussion:
> http://www.w3.org/TR/UNDERSTANDING-WCAG20/conformance.html#uc-accessibility-support-head
>
> Examples:
> Some organizations and tool vendors explain how to use various tools, but
> only includes some of the tools that have been decided upon to be
> standardize on based on some set of browsers and platform assumptions.  As
> new browsers and platforms are now no longer "emerging", but somewhat
> established with various levels of ARIA support for example; what is the
> current status and possibility for standardizing on them?  I believe this
> "research and documentation" should be a responsibility of WAI, just like
> the implementation techniques are documented - agreement?
>
> I need a reference to review if any of these screen readers are possible
> for establishing a "standard set" for accessibility verification test
> (AVT) - and of course why and why not?
>          NVDA on Windows 7 with IE browser?
>          ChromeVox on Windows 7 with the Chrome browser?
>          ChromeVox on MacOS with the Chrome browser - or should Safari with
> VoiceOver be the standard, does it matter, why?
>          Talkback on Android 5.0 with the _________ browser?
>
> We can't test on every tool, every platform, and every browser
> combination, so which set is the "best set" for a project's standard set?
> We need references documented.  We need to know what is being worked on
> and which tools are being researched?
>
> There is a great set of questions and criteria with which to make
> decisions, but there is no summarized referenced data by tool - we need
> the "answers" too.
>          see http://www.w3.org/WAI/eval/selectingtools.html
>
> So when Jen and others ask the question: which tools?  we can point them
> to a resource just like we can point developers to the techniques
> resources.
>
> The list of tools at http://www.w3.org/WAI/ER/tools/index.html needs to
> cross-indexed by platform, by Success Criteria, by browser, and by content
> technology to be more useful.
> ____________________________________________
> Regards,
> Phill Jenkins,
> IBM Accessibility
> http://www.ibm.com/able
> http://www.facebook.com/IBMAccessibility
> http://twitter.com/IBMAccess
> http://www.linkedin.com/in/philljenkins
>
>
>
> From:   Mike Elledge <melledge@yahoo.com>
> To:     Jens Oliver Meiert <jens@meiert.com>, W3C WAI IG
> <w3c-wai-ig@w3.org>
> Date:   12/09/2014 09:33 AM
> Subject:        Re: Fwd: Accessibility tools
>
>
>
> Hi Jens (all)--
>
> I use different tools depending on the browser (most common shown by *):
> IE: *Web Accessibility Toolbar, *WAVE
> Firefox: FAE, Web Developer Toolbar, WAVE, Juicy Studio Accessibility
> plug-ins
> Chrome: Web Developer Toolbar
> Screenreaders: *JAWS, NVDA, VoiceOver and ChromeVox
>
> Hope that helps!
>
> Mike
>
>
> On Tuesday, December 9, 2014 10:10 AM, Jens Oliver Meiert
> <jens@meiert.com> wrote:
>
>
> Forwarding per suggestion from Andrew?cheers!
>
>
> ---------- Forwarded message ----------
> From: Jens Oliver Meiert <jens@meiert.com>
> Date: Tue, Dec 9, 2014 at 11:46 PM
> Subject: Accessibility tools
> To: W3C WAI GL <w3c-wai-gl@w3.org>
>
>
> An informal question, what accessibility tools do people here trust
> most these days?
>
> I?m reviewing the accessibility section of UITest.com and am not sure
> it?s up-to-date.
>
> (Direct feedback okay if you happen to run any tools you like to see
> listed there, or if you have any general feedback.)
>
> --
> Jens Oliver Meiert
> http://meiert.com/en/
>
>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Tuesday, 9 December 2014 21:01:12 UTC