Re: Fwd: Accessibility tools

Very good to hear about those efforts, Shadi.  Also, I agree 100% with
the challenges you cite on automated testing.  For instance, claiming
a tool has 100% coverage with even 1.1.1 is impossible. There are many
automated checks which can look for conformance against 1.1.1 but the
ultimate judgment of a text alternative is whether it accurately
represents the non-text content in context of use.   That doesn't
reduce the usefulness of automation but does create considerable
challenges when creating the type of matrix described by Phill. :-(

On Tue, Dec 9, 2014 at 4:00 PM, Shadi Abou-Zahra <shadi@w3.org> wrote:
> Hi Phil,
>
> Thank you for this input -- very useful thoughts.
>
> Indeed, the WAI Evaluation and Repair Tools Working Group (ERT WG) has
> looked at several approaches in the past but it turned out to be not that
> trivial. For example, what does it mean when we say evaluation tool X
> addresses Success Criteria Y? Some tools will check the code to identify
> whether a particular Success Criteria is applicable, others might then do
> partial checks on the corresponding code (with varying degrees of
> conclusiveness), and yet others might just automatically list all Success
> Criteria "for manual verification" by the evaluator. It would be hard to
> draw a line and say which tool addresses specific Success Criteria without a
> more comprehensive set of test cases. Same problem arises for web technology
> coverage and other checking aspects.
>
> The "WCAG 2.0 Test Samples" is intended to be such a test suite:
>  - http://www.w3.org/WAI/ER/tests/
>
> (Much of that work was contributed by the EC-funded BenToWeb Project.)
>
> In parallel, the Auto-WCAG community group is also doing related work on
> defining automated "checks" for WCAG 2.0 rather than test cases:
>  - http://www.w3.org/community/groups/#auto-wcag
>
> (That work was initiated by another EC-funded project called EIII.)
>
> Another complimentary approach that we are taking is to describe the
> "features" that different tools could provide, which provides a way of
> better describing what particular tools can and cannot do:
>  - http://www.w3.org/TR/WAET/
>
> (This was developed with support from the EC-funded WAI-ACT project.)
>
> So, we have several pieces that we (the community) could build on to develop
> that matrix that you are talking about. It would require quite some effort
> though. I would be delighted to explore opportunities.
>
> Best,
>   Shadi
>
>
> On 9.12.2014 19:25, Phill Jenkins wrote:
>>
>> My recommendation:
>> I recommend a center of competency (e.g. W3C WAI Evaluation & Repair
>> Working Group) maintain a matrix like table of contents or
>> cross-referenced searchable index to document the current  and future
>> status for which accessibility test tools on which platforms for each WCAG
>> 2.0 Success Criteria.   There are at least two approaches to display this
>> reference - By WCAG 2.0 Success Criteria: this is what  tools are
>> available for each Success Criteria by platform and by browser; versus the
>> by platform view: this is the set of platforms and browsers an enterprise
>> may have already established, what is the status of accessibility tools?
>>
>> A screen reader is just one tool set but covers a major set of the 38
>> Level A and Double AA Success Criteria.  The automated checker tool(s) is
>> a second set of tools.  Color Contrast Analyzers and other browser toolbar
>> plug-ins are a third set of tools and often only cover a single or smaller
>> set of Success Criteria.
>>
>> Context: Often we accessibility practitioners get engaged about the
>> question: "what is the standard set of tools we should use to validate
>> accessibility?"  and we get engaged to deliver courses that include
>> training on tools for designers, developers, and testers.  How can the
>> enterprise be enables?  The budget for tools and training is often a
>> strong consideration in establishing the "standard tool set".  The browser
>> and platform standards at an organization is another strong consideration
>> for establishing the standard tool set, e.g. we hear a government agency
>> say: "but we have to test on IE 9, our standard desktop browser", or the
>> bank will say, "it has to run on Firefox ESR", etc.  The other strong
>> consideration is the chosen standard set of platforms; iOS 8, Android 5.0,
>> Windows 7, ChromeOS 40, etc.
>>
>> Note: I am NOT talking about the accessibility capital S Standard - based
>> on WCAG 2.0, but I'm talking about  the lower case standard set of
>> browser, platforms and tools that an enterprise or project standardizes on
>> . . .
>>
>> The WCAG working group alluded to this as the "accessibility supported"
>> discussion:
>>
>> http://www.w3.org/TR/UNDERSTANDING-WCAG20/conformance.html#uc-accessibility-support-head
>>
>> Examples:
>> Some organizations and tool vendors explain how to use various tools, but
>> only includes some of the tools that have been decided upon to be
>> standardize on based on some set of browsers and platform assumptions.  As
>> new browsers and platforms are now no longer "emerging", but somewhat
>> established with various levels of ARIA support for example; what is the
>> current status and possibility for standardizing on them?  I believe this
>> "research and documentation" should be a responsibility of WAI, just like
>> the implementation techniques are documented - agreement?
>>
>> I need a reference to review if any of these screen readers are possible
>> for establishing a "standard set" for accessibility verification test
>> (AVT) - and of course why and why not?
>>          NVDA on Windows 7 with IE browser?
>>          ChromeVox on Windows 7 with the Chrome browser?
>>          ChromeVox on MacOS with the Chrome browser - or should Safari
>> with
>> VoiceOver be the standard, does it matter, why?
>>          Talkback on Android 5.0 with the _________ browser?
>>
>> We can't test on every tool, every platform, and every browser
>> combination, so which set is the "best set" for a project's standard set?
>> We need references documented.  We need to know what is being worked on
>> and which tools are being researched?
>>
>> There is a great set of questions and criteria with which to make
>> decisions, but there is no summarized referenced data by tool - we need
>> the "answers" too.
>>          see http://www.w3.org/WAI/eval/selectingtools.html
>>
>> So when Jen and others ask the question: which tools?  we can point them
>> to a resource just like we can point developers to the techniques
>> resources.
>>
>> The list of tools at http://www.w3.org/WAI/ER/tools/index.html needs to
>> cross-indexed by platform, by Success Criteria, by browser, and by content
>> technology to be more useful.
>> ____________________________________________
>> Regards,
>> Phill Jenkins,
>> IBM Accessibility
>> http://www.ibm.com/able
>> http://www.facebook.com/IBMAccessibility
>> http://twitter.com/IBMAccess
>> http://www.linkedin.com/in/philljenkins
>>
>>
>>
>> From:   Mike Elledge <melledge@yahoo.com>
>> To:     Jens Oliver Meiert <jens@meiert.com>, W3C WAI IG
>> <w3c-wai-ig@w3.org>
>> Date:   12/09/2014 09:33 AM
>> Subject:        Re: Fwd: Accessibility tools
>>
>>
>>
>> Hi Jens (all)--
>>
>> I use different tools depending on the browser (most common shown by *):
>> IE: *Web Accessibility Toolbar, *WAVE
>> Firefox: FAE, Web Developer Toolbar, WAVE, Juicy Studio Accessibility
>> plug-ins
>> Chrome: Web Developer Toolbar
>> Screenreaders: *JAWS, NVDA, VoiceOver and ChromeVox
>>
>> Hope that helps!
>>
>> Mike
>>
>>
>> On Tuesday, December 9, 2014 10:10 AM, Jens Oliver Meiert
>> <jens@meiert.com> wrote:
>>
>>
>> Forwarding per suggestion from Andrew?cheers!
>>
>>
>> ---------- Forwarded message ----------
>> From: Jens Oliver Meiert <jens@meiert.com>
>> Date: Tue, Dec 9, 2014 at 11:46 PM
>> Subject: Accessibility tools
>> To: W3C WAI GL <w3c-wai-gl@w3.org>
>>
>>
>> An informal question, what accessibility tools do people here trust
>> most these days?
>>
>> I?m reviewing the accessibility section of UITest.com and am not sure
>> it?s up-to-date.
>>
>> (Direct feedback okay if you happen to run any tools you like to see
>> listed there, or if you have any general feedback.)
>>
>> --
>> Jens Oliver Meiert
>> http://meiert.com/en/
>>
>>
>>
>
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
> Activity Lead, W3C/WAI International Program Office
> Evaluation and Repair Tools Working Group (ERT WG)
> Research and Development Working Group (RDWG)
>



-- 

Karl Groves
www.karlgroves.com
@karlgroves
http://www.linkedin.com/in/karlgroves
Phone: +1 410.541.6829

Modern Web Toolsets and Accessibility
https://www.youtube.com/watch?v=_uq6Db47-Ks

www.tenon.io

Received on Tuesday, 9 December 2014 22:25:21 UTC