RE: Deque Axe-Core and WCAG mappings

My view is that it’s one huge gap and that axe-core does not eliminate the need for any manual testing at all. It only partially tests most success criteria, and it fully tests so few that it’s not worth trying to work out which they are. And then there are all the false positives that need to be verified manually.

All automated tools are like this. Tools designed for use with continuous integration, like Tenon, give fewer false positives (ideally none) but they find even fewer real faults than other tools (because they rely on rules rather than heuristics).

We only use automated tools as a safety net after we have done a full manual test, in case we missed anything. I would advise others to do the same. We don’t even find them useful for supporting manual testing while we are doing it – in fact we find them a distraction. We use a load of single-purpose bookmarklets and other tools to improve efficiency instead.

Steve Green
Managing Director
Test Partners Ltd


From: John Foliot <john@foliot.ca>
Sent: 29 September 2021 18:03
To: Lisette Arocha (US) <lisette.m.arocha@pwc.com>
Cc: Wai-Ig <w3c-wai-ig@w3.org>
Subject: Re: Deque Axe-Core and WCAG mappings

Try here: https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md


JF

On Wed, Sep 29, 2021 at 12:45 PM Lisette Arocha (US) <lisette.m.arocha@pwc.com<mailto:lisette.m.arocha@pwc.com>> wrote:
Hi everyone,

I was wondering if anyone has done any work to map out which axe-core rules have good coverage of WCAG success criteria and where the gaps exist for manual testing.

For example, axe-core will pick up where there is no alt attribute but can't determine if the alt text is meaningful, if the image is decorative and should have an empty alt. Additionally, because it only flags the images that have no alt, is there a way to surface what images exist on a site with alt text to help speed up manual testing which would have to check if it's meaningful and/or decorative.

Lisette Maria Arocha
PwC | Manager |  Accessibility & Inclusive Design
Governance, Privacy & Ethics (GP&E) for Products & Technology
Miami | +1 (786) 599 7431
Pronouns: she/her/hers
CPWA, CSM



The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you received this in error, please contact the sender and delete the material from any computer.



In the event the content of this email includes Tax advice, the content of this email is limited to the matters specifically addressed herein and is not intended to address other potential tax consequences or the potential application of tax penalties to this or any other matter.



PricewaterhouseCoopers LLP is a Delaware limited liability partnership.  This communication may come from PricewaterhouseCoopers LLP or one of its subsidiaries.


--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"