Re: Question about proper use of screen readers in 508 testing

Well, I’ve learned a lot from this thread. Thank you all. I clearly have
some things to learn about accessibility.



I had not heard of NVDA before. My own team uses JAWS because this is the
best tool we knew about. I have no problem with fixing usability problems
(I want to!); I just don’t want them rated as critical and putting my
schedule at risk when they’re not critical.



I had not heard of such an auditory processing condition before. I don’t
want our testers logging bugs because JAWS is supposed to show some widget
in its list of form fields, regardless of actual accessibility. It seems
trivial and makes me wonder what we’re missing. But I also didn’t know that
expand/collapse widgets that don’t have the role of “button” or “link” are
WCAG non-conformances.



For me, it all just underscores the need to understand the WCAG standards
and the bewildering variety of circumstances that users face. The hard part
will be trying to get the certain people to look up from the tool and see
the bigger picture. Some here take a hard line on “button button” because
it was found as part of 508 testing, and 508 compliance is a contractual
obligation, so that’s that. I disagree about that, but I don’t really have
the authority when it comes to accessibility; they do.


Mike

On Thu, Jul 23, 2020 at 6:52 PM Steve Green <steve.green@testpartners.co.uk>
wrote:

> The situation is very simple, but sadly a lot of testers make the same
> mistake that this team is making. Tools simply point to the possible
> existence of a WCAG non-conformance. The pass / fail decision should always
> be made by the tester, based on inspection of the code and the user
> interface. Tools also fail to identify most issues, so testers need the
> skills to find those issues themselves. You are right that a lot of testers
> (perhaps most of them) report based on the output of a tool or assistive
> technology, perhaps because they don’t understand the code sufficiently
> well. If that is the case, they need to be trained or replaced.
>
>
>
> In the case of the expand/collapse widgets, it sounds like the clickable
> areas do not have a role of “button” or “link”, which would indeed be a
> WCAG non-conformance. It can easily be verified by checking the code. The
> expanded / collapsed state should also be conveyed programmatically. Note
> that issues like this can easily be found by code inspection - you don’t
> need a screen reader to find them.
>
> As Patrick said in his reply, there is clearly a training need if the
> testers think that “button button” is any kind of problem, let alone a
> critical one. And it’s definitely not a WCAG non-conformance – at most it
> warrants a comment. Screen reader users have to put up with far worse than
> that, and they mostly ignore anything they don’t understand.
>
>
>
> There are pros and cons to using JAWS rather than NVDA or other screen
> readers in a WCAG audit. In fact, it’s not necessary to use a screen reader
> at all, although it can be useful. JAWS used to have the overwhelming
> market share, but recent surveys suggest that NVDA is now close to parity
> with it. For WCAG testing purposes I would recommend using NVDA because it
> gives a relatively unadulterated reflection of the coding, although it has
> some weirdnesses. By contrast, JAWS uses a lot of heuristics to improve the
> user experience, so it hides some WCAG non-conformances and can cause some
> odd behaviours that will not occur with other screen readers.
>
>
>
> It is important to understand that a WCAG audit has nothing to do with
> screen reader behaviour. As Patrick said, the pass / fail decision should
> be based on WCAG’s normative requirements (with guidance from the
> supporting non-normative documents). You may choose to also do assistive
> technology testing, in which case you can report any behaviours you like,
> but that is different from a WCAG audit, which has a clearly defined scope.
>
>
>
> Steve Green
>
> Managing Director
>
> Test Partners Ltd
>
>
>
>
>
> *From:* Mike Cleary <mike.cleary@grantsolutions.gov>
> *Sent:* 23 July 2020 16:27
> *To:* w3c-wai-ig@w3.org
> *Subject:* Question about proper use of screen readers in 508 testing
>
>
>
> Hello,
>
> I'm new to the discussion list and have a question about how much reliance
> should be accorded to screen readers like JAWS when reporting accessibility
> issues.
>
>
>
> We have an internal testing team that uses JAWS for 508 testing. They are
> reporting accessibility issues in cases where JAWS reads all the content on
> screen, but does not recognize certain expand/collapse widgets as clickable
> links.
>
>
>
> In a different case, they have filed a "critical" bug in cases where a
> button is read as "button button.." Using the button is no problem; their
> argument is that the duplicate listing is potentially confusing. I say
> that's a usability problem, not an accessibility issue and thus not
> critical.
>
>
>
> My concern is that the testers are testing to the tool, not to
> accessibility guidelines. Am I mistaken? Is there any guidance on how much
> to rely upon a tool? Is there anything in WCAG 2.0 that speaks to this
> issue?
>
>
>
> Mike
>