Re: Disclosure Navigation Menu Example Test Plan Feedback

Hi Alyssa,

This is great feedback and super clear.  Hadi participated directly in a discussion of all four points on today's CG call, but to provide a summary:

1. Focus does seem to go to, and stay on, the button to run the test setup script.  However, the "automatic say all" that JAWS does on page load by default makes it appear that this isn't the case.  Users can actually activate the button with Enter/Space, even when JAWS has announced the end of the main landmark.  However, the button can also be explicitly navigated to if required.

2. I have taken on the task to fix/improve the testing instructions, which were not updated after implementation of the new setup button.  I'll also try to make point #1 more clear when doing so.

3. Not sure what would cause this, if you're always starting the test from the same defined point.  Please list any inconsistencies within the "unexpected results" box, and they will be manually investigated once multiple people have executed the same test.

4. The instructions relating to interaction mode, i.e. informing the tester how to activate the JAWS PC Cursor in this case, are generated by the test renderer whenever the test mode is set to "interaction".  They are not included in each test explicitly, to avoid inconsistencies across tests, but they are repeated for the tests where they apply.  I verified that the instructions are in place for this specific test at:

https://w3c.github.io/aria-at/build/review/disclosure-navigation.html

If you run into a similar problem, or the location/content of the instructions is unclear, please let us know.

And finally: specifically following discussion of point #3, we revisited the fact that there are currently no explicit instructions informing a tester how to copy screen reader output, and that it shouldn't be done by hand if at all possible.  I've created an issue (https://github.com/w3c/aria-at/issues/534) to track this.

Thanks again!

Regards,

James Scholes
Director of Digital Accessibility, Prime Access Consulting, Inc.

On 21/10/2021 at 2:07 pm, Alyssa Vo wrote:
> Hi everyone,
> 
>  
> 
> Hadi and I spent some time this morning to continue our testing of some of the test plans. Below is our feedback as we went through tasks 2, 3, and 4 of “Disclosure Navigation Menu Example” with JAWS and Chrome.
> 
>   * Upon activating the “open test page” button, with JAWS, focus is not moved to the “Run Test Setup” button. Instead, you are taken to the end of the main region. We believe that this is due to the window not being fully loaded before JAWS starts talking. With keyboard only, focus is correctly moved to the “Run Test Setup” button.
>   * The instructions do not explicitly tell the user to press the “Run Test Setup” button. Instead, the instructions say that the window “runs a script that sets focus on the link.” This implies that the script runs automatically.
>   * We ran into some inconsistent announcements in task 2 while using shift+tab. During our first run through, it announced “mythical university navigation region list with three items about button collapsed to activate press enter”. However, we could not reproduce this output the second time. The second time JAWS only announced “about button collapsed”. I entered this feedback in my results for task 2.
>   * The instructions for task 4 do not include a specific step informing the user to switch JAWS to interactive mode. It’s implicitly communicated through the title, but not written out in the instructions.
> 
>  
> 
> If anyone has questions about our feedback, please let us know. We’d be happy to elaborate or demo as needed.
> 
>  
> 
> Thanks,
> 
> Alyssa
> 
>  
> 
> Alyssa Vo
> 
> /She/her/hers/
> 
> Student Accessibility Assistant, UW-IT
> 
> 425-633-0845
> 
> _avo4@uw.edu <mailto:avo4@uw.edu>_
> 
>  
>

Received on Thursday, 21 October 2021 22:05:55 UTC