Re: requirements to test with Jaws in different organizations

A lot of good points have been raised here. I'll add another issue to be
aware of with regard to testing with a specific piece of software, and here
JAWs.

It has been my observation that in environments where software and
computing devices are maintained or supported centrally (i.e. a corporate
IT Dept. or similar) that for those organizations, they'll likely have JAWs
installed on their corporate machines, for the simple reason that a license
for software like that usually comes with some kind of "1-800" service
department for when 'issues' arise.

Corporate IT departments are rarely AT specialists (so they need that kind
of turn-key assistance), whereas with an Open Source tool like NVDA, if you
encounter an issue, you're pretty much stuck with turning to community
forums (etc.) when things aren't working as expected, which is often not
viable in a business context. (I'll note that this is generally true for
*any* Open Source software, and is not limited to NVDA).

As others have noted, inserting a screen reader into the WCAG conformance
testing effort is often useful, but generally not a *requirement*. However,
just as QA specialists will (at some point) test with multiple browsers
prior to 'launch', testing your content with more than one screen reader is
another step in ensuring your content works as expected for all, and I
would strongly urge any organization to add screen readers to their
User-Acceptance Testing (UAT) protocols. Given the market share that JAWs
continues to hold today, I'd strongly recommend that JAWs be used in that
UAT pass.

JF

On Thu, Jul 8, 2021 at 10:32 AM Chris O'Brien <chrobrien@olg.ca> wrote:

> Hi Mitchell,
>
>
>
> I can appreciate both sides of the argument, but to your last point, code
> implementation often requires understanding how it will be supported by
> browsers and can be checked via sites like caniuse.com. I don’t see
> accessibility being much different in this regard. Devs and testers can
> access support matrices for ARIA support with sites like a11ysupport.io
> and Powermapper <https://www.powermapper.com/tests/screen-readers/aria/>
> in a similar manner.
>
>
>
> From my perspective, people may have honed their technique in various ways
> and can arrive at similar conclusions using different methodologies. I
> don’t think this is a one size fits all scenario.
>
>
>
> Chris O’Brien
>
> Director of Accessibility
>
> Legal and Litigation
>
>
>
>
>
> OLG Internal
>
> *From:* Mitchell Evan <mtchllvn@gmail.com>
> *Sent:* Wednesday, July 7, 2021 4:11 AM
> *To:* Rajiv Shah <rmshah@starpower.net>
> *Cc:* Annie Heckel <annieh@onlineada.com>; WAI IG <w3c-wai-ig@w3.org>
> *Subject:* Re: requirements to test with Jaws in different organizations
>
>
>
> This email originated outside of OLG. Do not open attachments or click
> links unless you recognize the sender and know the content is safe.
>
> > If you wish to test for usability with a screen reader, then please go
> ahead. I think this is fine. But for WCAG testing?? I am not sure if AT and
> testing for standards viability makes sense.
>
>
>
> It's true we can tell a lot from DOM inspection and tools, but code
> quality alone is not quite enough. For example, a coding technique which
> relies on aria-controls or aria-owns could pass all code validators yet
> fail the WCAG "accessibility supported" requirement
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F%23cc4&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318691124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AQ4REWtIE%2BscpgG46fl5OE5Am9KHzFs2F3Xmhk7r%2FEo%3D&reserved=0>
> because the technique doesn't actually work in currently available
> assistive technologies.
>
>
>
> Another valid concern is that testers who are new to screen readers often
> misunderstand screen reader behaviors, leading to false positives and false
> negatives in WCAG criteria such as 1.3.1 and 4.1.3. However, it would be
> even more unfair to expect that same person, who has never used a screen
> reader, to examine only the code quality of a complex web application and
> predict whether it works well enough to meet these criteria.
>
>
>
> The answer is a judicious mix of code quality assessment and
> functional testing, which in the long run is faster and more accurate than
> either method alone.
>
>
>
> Mitchell Evan, CPWA
>
> linkedin.com/in/mitchellrevan
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmitchellrevan&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318701125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Hsa6KTz2TZCQ97SDe884Fs1mr7wA70CRXSp7iBio%2BDs%3D&reserved=0>
>
> Twitter @mitchellrevan
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fmitchellrevan&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318711121%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VOwoTK0YX0SaZUgUVV0Fr%2FdVdXjKb8uhywRAfseeeEA%3D&reserved=0>
>
> +49 1525 8950540
>
> +1 510 375 6104
>
>
>
>
>
> On Wed, 7 Jul 2021 at 03:58, Rajiv Shah <rmshah@starpower.net> wrote:
>
> Hi, Even though I am a screen-reader user, I disagree with the approach of
> testing with any specific AT. Why? Because JAWS has so many scripts which
> provide a lot of info on a page that goes beyond WCAG requirements. In
> short, it removes the standardization from the standard. If you wish to
> test for usability with a screen reader, then please go ahead. I think this
> is fine. But for WCAG testing?? I am not sure if AT and testing for
> standards viability makes sense. Thank you. Rajiv
>
> ----- Original Message -----
> From: Annie Heckel <annieh@onlineada.com>
> To: WAI IG <w3c-wai-ig@w3.org>
> Sent: Tue, 06 Jul 2021 10:55:01 -0400 (EDT)
> Subject: Re: requirements to test with Jaws in different organizations
>
> Bryan,
>
>
>
> The main reason that I can think of to test with JAWS *as well as* NVDA
> and Voice Over is that JAWS was the most popular screen reader, by far, in WebAIM's most recent
> Screen Reader User survey
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwebaim.org%2Fprojects%2Fscreenreadersurvey9%2F&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318711121%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kwqDnEEHtY%2FObHUmCN0hmRruTq6munqrZi9Ra6grzxs%3D&reserved=0>.
> It should not add significant time to the testing process, in my
> experience. My team does not usually go through the entire website with
> screen readers, but instead uses them for spot checks when there's an odd
> structure and we're not sure how it will work with screen readers, and to
> create screencasts to illustrate issues such as multiple links that have
> the same accessible text. As Steve noted, most WCAG conformance problems
> can be identified by other means; I have found that learning to navigate
> with a screen reader was most valuable for helping me know what problematic
> code patterns to look for in relation to specific kinds of webpage
> structures.
>
>
>
> -Annie
>
> On Mon, Jul 5, 2021 at 1:57 AM bryan rasmussen <rasmussen.bryan@gmail.com>
> wrote:
>
> Hi,
>
>
> This might be slightly off for the list, but don't know where else to
>
> ask that would be good. So I am working on a project for an online
>
> education platform that wants to move into the U.S market.
>
>
> The platform would like to limit testing at least part of the
>
> development process to nvda and voiceover, to decrease costs /
>
> workload, I personally would like to test with -Jaws all the time but
>
> often one can't for various reasons. So does anyone have a pattern for
>
> how they do periodic testing with Jaws in such a way as to decrease
>
> workload). And more importantly if anyone knows of any organizational
>
> / governmental regulations requiring testing with Jaws that would
>
> affect the educational market in the U.S?
>
>
> Thanks,
>
> Bryan Rasmussen
>
>
>
>
> --
>
> Annie Heckel
>
>
>
> Lead Accessibility Auditor
>
> Online ADA
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fonlineada.com%2F&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318721112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DJqBhIwx5BWEIdJMiU7qbKXUwse%2Fz9KeNdyrWzmV10c%3D&reserved=0>
>
>

-- 
*John Foliot* | Senior Industry Specialist, Digital Accessibility

"I made this so long because I did not have time to make it shorter." -
Pascal "links go places, buttons do things"