RE: Global Shortcut Keys and Screen Reader Accessibility?

Taliesin, this sounds like some amazing work!

 

Matt

 

-----Original Message-----
From: Taliesin Smith <talilief@gmail.com> 
Sent: Tuesday, April 28, 2020 5:59 AM
To: Matt King <a11ythinker@gmail.com>
Cc: Karen Lewellen <klewellen@shellworld.net>; WAI IG <w3c-wai-ig@w3.org>
Subject: Re: Global Shortcut Keys and Screen Reader Accessibility?

 

Hi Matt,

I agree, it will be awesome to make the hotkey customizable. I hope to make that part of future research.  

 

For now we’ve got Alt + k implemented as the hotkey to toggle between paused and playing states no matter where the learner’s current keyboard focus is. 

 

Basically, once the action in the simulation has been started, i.e. once the photons are firing at the molecule; the hotkey combination can be used to pause and play the action without the learner needing to move their focus to the actual Pause/Play button.

 

As for role application, we only use it for custom interactions where a native HTML option is not available. For example, we have successfully used role application to make a grabbed balloon moveable in four directions. As the learner grabs the balloon with a native button interaction, role application is launched creating a 4-way custom drag interaction, enabling a learner to move the grabbed balloon in any direction. The learner can then use the arrow keys (or alternative letter keys WASD) to move the balloon to rub it against a sweater or a wall to explore how static electricity works. The balloon is released in the same way to was grabbed.

 

From the beginning of our research on making interactive science simulations accessible, we treaded very carefully with role application. Our research with users tells us that our native and custom interactions are quite intuitive and accessible to blind learners. That said, we have found, thus far, that starting a custom interaction with a native one, like a button, works better for blind learners than immediately presenting a custom interaction upon focus.

 

The suggestions, advice, comments, perspectives I am exposed to on this list are awesome.

 

Taliesin

 

~.~.~

Available off list at  <mailto:Taliesin.Smith@colorado.edu> Taliesin.Smith@colorado.edu Inclusive Design Researcher PhET Interactive Simulations  <https://phet.colorado.edu/en/accessibility> https://phet.colorado.edu/en/accessibility

Physics Department

University of Colorado, Boulder

 

> On Apr 28, 2020, at 4:18 AM, Matt King < <mailto:a11ythinker@gmail.com> a11ythinker@gmail.com> wrote:

> 

> Customizable is definitely good.

> 

> I'd recommend avoid using role application. Please carefully read the spec for role application before deciding whether to go down that path.

> 

> Matt

> 

> -----Original Message-----

> From: Taliesin Smith < <mailto:talilief@gmail.com> talilief@gmail.com>

> Sent: Wednesday, April 15, 2020 1:33 PM

> To: Karen Lewellen < <mailto:klewellen@shellworld.net> klewellen@shellworld.net>

> Cc: WAI IG < <mailto:w3c-wai-ig@w3.org> w3c-wai-ig@w3.org>

> Subject: Re: Global Shortcut Keys and Screen Reader Accessibility?

> 

> Hi Karen,

> Great question. The answer, unfortunately is no.

> 

> For our simulations (PhET Interactive Simulations) we test in these three pairs (VO & Safari, NVDA & FF, and JAWS & Chrome). That said, while presenting at the CSUN conference a couple of years ago a person in the audience tweeted that the simulation I was demoing worked great in Orca on Linux.

> 

> I agree, the challenge is finding an intuitive combination that does conflict with anything else.

> 

> I liked Jonathan’s idea of making the shortcut customizable. We are currently doing research on adaptive features.

> 

> Taliesin

> 

>> On Apr 15, 2020, at 5:48 PM, Karen Lewellen < <mailto:klewellen@shellworld.net> klewellen@shellworld.net> wrote:

>> 

>> Do you intend testing with Linux and its various  screen reader programs?

>> Your goals are terrific, with my one concern being finding a key combination that is not used by  one of the scores of screen readers a person might choose  for their individual accommodating needs.

>> Karen

>> 

>> 

>> 

>> On Wed, 15 Apr 2020, Taliesin Smith wrote:

>> 

>>> Hi Folks,

>>> I am looking into the feasibility of global hotkey or Keyboard Shortcut that would toggle a Pause/Play button in a simulated learning environment regardless of where the keyboard focus is currently at in the application.

>>> 

>>> While the Pause/Play button interaction is somewhat akin to how a video player works, it is not exactly the same kind of experience in our situation.

>>> 

>>> We are currently working on a science simulation called Molecules and Light < <https://phet-dev.colorado.edu/html/molecules-and-light/1.5.0-dev.44/phet/molecules-and-light_en_phet.htm> https://phet-dev.colorado.edu/html/molecules-and-light/1.5.0-dev.44/phet/molecules-and-light_en_phet.htm> (note to hear the interactive descriptions you need to use a screen reader - VO & Safari, NVDA & FF or JAWS & Chrome).

>>> 

>>> In this simulated environment a learner turns a light source to fire photons at a molecule. The learner can change the light source and the molecule on the fly and observe what happens.

>>> 

>>> Learners can also Pause the action (the firing of photons) without turning the light source off. When the action is paused, the learner can use a Step Forward button to move the action (the photons) forward little by little to examine more closely what happens when a photon passes through or is absorbed by the molecule.

>>> 

>>> In interviews with blind learners, we found that learners were confused by the Pause/Play and Step Forward buttons, especially when the light source had not already been turned on. And if the simulation was set to Pause, they were often temporarily confused why the light source button would no longer fire photons.

>>> 

>>> We have made improvements to the descriptive experience by adding aria-live alerts that indicate if the light source is off or if the sim is paused, and why it might be useful to use the Step Forward button when the Pause button is activated. We think these aria-live alerts will create a much more intuitive experience (more user testing is pending).

>>> 

>>> That said, a few learners requested a global keyboard shortcut to quickly Pause and Play the action no matter where their focus was. My question to the list is, does any one have any experience or resources on making a global keyboard shortcut that does not rely on focus and works when screen reader software is enabled. I am assuming we would need to use a modifier key. Something like Control + k or Alt + k.

>>> 

>>> This kind of Pause/Play and Step Forward interaction is used in many of our interactive science simulations. A global hot key could be useful for all learners whether they use screen readers or not.

>>> 

>>> Thanks in advance for any resources on accessible global keyboard shortcuts. I have read this great medium article by Sasha Maximova, J, K, or How to choose keyboard shortcuts for web applications < <https://medium.com/@sashika/j-k-or-how-to-choose-keyboard-shortcuts-for-web-applications-a7c3b7b408ee> https://medium.com/@sashika/j-k-or-how-to-choose-keyboard-shortcuts-for-web-applications-a7c3b7b408ee>, several times, and am looking for more resources.

>>> 

>>> Taliesin

>>> ~.~.~

>>> Also available off list at:  <mailto:Taliesin.Smith@colorado.edu> Taliesin.Smith@colorado.edu Inclusive 

>>> Design Researcher PhET Interactive Simulations 

>>>  <https://phet.colorado.edu/en/accessibility> https://phet.colorado.edu/en/accessibility

>>> Physics Department

>>> University of Colorado, Boulder

>>> 

>>> 

>>> 

>>> 

>>> 

>>> 

>>> 

> 

> 

>