RE: requirements to test with Jaws in different organizations

Hi Mitchell,

I can appreciate both sides of the argument, but to your last point, code implementation often requires understanding how it will be supported by browsers and can be checked via sites like caniuse.com<https://caniuse.com/>. I don't see accessibility being much different in this regard. Devs and testers can access support matrices for ARIA support with sites like a11ysupport.io<https://a11ysupport.io/> and Powermapper<https://www.powermapper.com/tests/screen-readers/aria/> in a similar manner.

From my perspective, people may have honed their technique in various ways and can arrive at similar conclusions using different methodologies. I don't think this is a one size fits all scenario.

Chris O'Brien
Director of Accessibility
Legal and Litigation



OLG Internal
From: Mitchell Evan <mtchllvn@gmail.com>
Sent: Wednesday, July 7, 2021 4:11 AM
To: Rajiv Shah <rmshah@starpower.net>
Cc: Annie Heckel <annieh@onlineada.com>; WAI IG <w3c-wai-ig@w3.org>
Subject: Re: requirements to test with Jaws in different organizations

This email originated outside of OLG. Do not open attachments or click links unless you recognize the sender and know the content is safe.
> If you wish to test for usability with a screen reader, then please go ahead. I think this is fine. But for WCAG testing?? I am not sure if AT and testing for standards viability makes sense.

It's true we can tell a lot from DOM inspection and tools, but code quality alone is not quite enough. For example, a coding technique which relies on aria-controls or aria-owns could pass all code validators yet fail the WCAG "accessibility supported" requirement<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F%23cc4&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318691124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AQ4REWtIE%2BscpgG46fl5OE5Am9KHzFs2F3Xmhk7r%2FEo%3D&reserved=0> because the technique doesn't actually work in currently available assistive technologies.

Another valid concern is that testers who are new to screen readers often misunderstand screen reader behaviors, leading to false positives and false negatives in WCAG criteria such as 1.3.1 and 4.1.3. However, it would be even more unfair to expect that same person, who has never used a screen reader, to examine only the code quality of a complex web application and predict whether it works well enough to meet these criteria.

The answer is a judicious mix of code quality assessment and functional testing, which in the long run is faster and more accurate than either method alone.

Mitchell Evan, CPWA
linkedin.com/in/mitchellrevan<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmitchellrevan&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318701125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Hsa6KTz2TZCQ97SDe884Fs1mr7wA70CRXSp7iBio%2BDs%3D&reserved=0>
Twitter @mitchellrevan<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fmitchellrevan&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318711121%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VOwoTK0YX0SaZUgUVV0Fr%2FdVdXjKb8uhywRAfseeeEA%3D&reserved=0>
+49 1525 8950540
+1 510 375 6104


On Wed, 7 Jul 2021 at 03:58, Rajiv Shah <rmshah@starpower.net<mailto:rmshah@starpower.net>> wrote:
Hi, Even though I am a screen-reader user, I disagree with the approach of testing with any specific AT. Why? Because JAWS has so many scripts which provide a lot of info on a page that goes beyond WCAG requirements. In short, it removes the standardization from the standard. If you wish to test for usability with a screen reader, then please go ahead. I think this is fine. But for WCAG testing?? I am not sure if AT and testing for standards viability makes sense. Thank you. Rajiv

----- Original Message -----
From: Annie Heckel <annieh@onlineada.com<mailto:annieh@onlineada.com>>
To: WAI IG <w3c-wai-ig@w3.org<mailto:w3c-wai-ig@w3.org>>
Sent: Tue, 06 Jul 2021 10:55:01 -0400 (EDT)
Subject: Re: requirements to test with Jaws in different organizations
Bryan,

The main reason that I can think of to test with JAWS as well as NVDA and Voice Over is that JAWS was the most popular screen reader, by far, in WebAIM's most recent Screen Reader User survey<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwebaim.org%2Fprojects%2Fscreenreadersurvey9%2F&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318711121%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kwqDnEEHtY%2FObHUmCN0hmRruTq6munqrZi9Ra6grzxs%3D&reserved=0>. It should not add significant time to the testing process, in my experience. My team does not usually go through the entire website with screen readers, but instead uses them for spot checks when there's an odd structure and we're not sure how it will work with screen readers, and to create screencasts to illustrate issues such as multiple links that have the same accessible text. As Steve noted, most WCAG conformance problems can be identified by other means; I have found that learning to navigate with a screen reader was most valuable for helping me know what problematic code patterns to look for in relation to specific kinds of webpage structures.

-Annie
On Mon, Jul 5, 2021 at 1:57 AM bryan rasmussen <rasmussen.bryan@gmail.com<mailto:rasmussen.bryan@gmail.com>> wrote:
Hi,


This might be slightly off for the list, but don't know where else to

ask that would be good. So I am working on a project for an online

education platform that wants to move into the U.S market.


The platform would like to limit testing at least part of the

development process to nvda and voiceover, to decrease costs /

workload, I personally would like to test with -Jaws all the time but

often one can't for various reasons. So does anyone have a pattern for

how they do periodic testing with Jaws in such a way as to decrease

workload). And more importantly if anyone knows of any organizational

/ governmental regulations requiring testing with Jaws that would

affect the educational market in the U.S?


Thanks,

Bryan Rasmussen


--
Annie Heckel

Lead Accessibility Auditor
Online ADA<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fonlineada.com%2F&data=04%7C01%7Cchrobrien%40olg.ca%7C6b036965687343a9d03808d9421b4e2a%7Cf271d9b4e54c46e182bd25d50afa3779%7C0%7C0%7C637613507318721112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DJqBhIwx5BWEIdJMiU7qbKXUwse%2Fz9KeNdyrWzmV10c%3D&reserved=0>