Re: Deque Axe-Core and WCAG mappings

Hi Steve,

While I no longer work at Deque, I'd like to seriously push back on your
comments regarding false-positives coming from axe-core. *Do you have any
evidence of axe-core generating false-positives today*?

From my 5+ years at Deque, NO FALSE POSITIVES was practically tattooed on
the foreheads of the developers who worked on axe-core, and so I reject
that comment on the surface, without evidence to the contrary.
Deque's Manifesto states:


   - Automated accessibility testing rules must have a zero false positive
   rate
   - Automated accessibility testing rules must be lightweight and fast
   - Automated accessibility testing rules must work in all modern browsers
   - Automated accessibility testing rules must, themselves, be tested
   automatically

Additionally, axe-core never claimed to "...*eliminate the need for any
manual testing at all*" - in fact, one of Deque's tools (axe Auditor
<https://www.deque.com/axe/>) merges both mechanical and manual testing
processes and results, *and facilitates the additional manual testing
process from within the tool *(Complete with instructions on how to test,
references to WCAG documentation, a standardized reporting language
library, and more). Depending on the page and page content however,
axe-core/the axe browser extension(s) can capture up to 50% or more of WCAG
2.x issues (and for example, in a content-free CMS template, quite likely
even more than 50%).

Finally, I will also note that axe-core CAN BE integrated into production
systems (CI/CD integrations <https://github.com/dequelabs/axe-core-gems>),
and that was always part of the larger development roadmap for axe-core, so
I am not sure why you believe otherwise.

Might I respectfully suggest that if you are going to comment on an
auditing tool, that you know the tooling of which you comment on? TIA!

JF

On Wed, Sep 29, 2021 at 1:25 PM Steve Green <steve.green@testpartners.co.uk>
wrote:

> My view is that it’s one huge gap and that axe-core does not eliminate the
> need for any manual testing at all. It only partially tests most success
> criteria, and it fully tests so few that it’s not worth trying to work out
> which they are. And then there are all the false positives that need to be
> verified manually.
>
>
>
> All automated tools are like this. Tools designed for use with continuous
> integration, like Tenon, give fewer false positives (ideally none) but they
> find even fewer real faults than other tools (because they rely on rules
> rather than heuristics).
>
>
>
> We only use automated tools as a safety net after we have done a full
> manual test, in case we missed anything. I would advise others to do the
> same. We don’t even find them useful for supporting manual testing while we
> are doing it – in fact we find them a distraction. We use a load of
> single-purpose bookmarklets and other tools to improve efficiency instead.
>
>
>
> Steve Green
>
> Managing Director
>
> Test Partners Ltd
>
>
>
>
>
> *From:* John Foliot <john@foliot.ca>
> *Sent:* 29 September 2021 18:03
> *To:* Lisette Arocha (US) <lisette.m.arocha@pwc.com>
> *Cc:* Wai-Ig <w3c-wai-ig@w3.org>
> *Subject:* Re: Deque Axe-Core and WCAG mappings
>
>
>
> Try here:
> https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md
>
> JF
>
>
>
> On Wed, Sep 29, 2021 at 12:45 PM Lisette Arocha (US) <
> lisette.m.arocha@pwc.com> wrote:
>
> Hi everyone,
>
>
>
> I was wondering if anyone has done any work to map out which axe-core
> rules have good coverage of WCAG success criteria and where the gaps exist
> for manual testing.
>
>
>
> For example, axe-core will pick up where there is no alt attribute but
> can't determine if the alt text is meaningful, if the image is decorative
> and should have an empty alt. Additionally, because it only flags the
> images that have no alt, is there a way to surface what images exist on a
> site with alt text to help speed up manual testing which would have to
> check if it's meaningful and/or decorative.
>
>
> *Lisette* *Maria* *Arocha*
>
> PwC | Manager |  Accessibility & Inclusive Design
>
> Governance, Privacy & Ethics (GP&E) for Products & Technology
>
> Miami | +1 (786) 599 7431
>
> Pronouns: she/her/hers
>
> CPWA, CSM
>
>
>
> The information transmitted, including any attachments, is intended only
> for the person or entity to which it is addressed and may contain
> confidential and/or privileged material. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient
> is prohibited, and all liability arising therefrom is disclaimed. If you
> received this in error, please contact the sender and delete the material
> from any computer.
>
>
>
> In the event the content of this email includes Tax advice, the content of
> this email is limited to the matters specifically addressed herein and is
> not intended to address other potential tax consequences or the potential
> application of tax penalties to this or any other matter.
>
>
>
> PricewaterhouseCoopers LLP is a Delaware limited liability partnership.
> This communication may come from PricewaterhouseCoopers LLP or one of its
> subsidiaries.
>
>
>
>
> --
>
> *John Foliot* |
> Senior Industry Specialist, Digital Accessibility |
> W3C Accessibility Standards Contributor |
>
> "I made this so long because I did not have time to make it shorter." -
> Pascal "links go places, buttons do things"
>


-- 
*John Foliot* |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |

"I made this so long because I did not have time to make it shorter." -
Pascal "links go places, buttons do things"