RE: Bi-modal touch interaction, possible with Voice Over enabled?

I am not an ARIA expert, but I am not aware of anything like that in iOS.  There are some so-called “split taps” where you hold down one finger, and tap with a second – but they are for the same control – not two different areas simultaneously.

Dave



From: Taliesin Smith <talilief@gmail.com>
Sent: Thursday, July 9, 2020 10:30 AM
To: w3c-wai-ig@w3.org
Subject: Bi-modal touch interaction, possible with Voice Over enabled?

This message may be from an external email source.
Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center.

________________________________
Hi Folks,

Do any aria experts on this list know if it is possible to create a custom touch interaction where 2 interaction points would be operational simultaneously on an iOS device using VoiceOver?

The reason I ask is that this is can quite easily happen on a touch device when VoiceOver is not invoked.

Basically, I am wondering if a user using VoiceOver can use two fingers at the same time to operate separate objects.

For context, we work on interactive science and math simulations. For an example, you can have a look at Gravity Force Lab Basics<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fphet.colorado.edu%2Fen%2Fsimulation%2Fgravity-force-lab-basics&data=02%7C01%7Cdavid.b.andrews%40state.mn.us%7Ca077a065c50d4a2f5f3008d8241dc211%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637299057517683932&sdata=C1xIzmhBT2m3Wzn3aFucpSrG1KHgYzV0GeUC41PuysE%3D&reserved=0>. This interactive simulation has two mass spheres. When a screen reader is not invoked, a learner can actually place one finger on mass 1 (the blue sphere) and another finger on mass 2 (the red sphere), and move the two spheres at the same time.

On iOS with VoiceOver invoked, learners are restricted to move one sphere at a time with VoiceOver’s swipes and gestures.

For this simulation, that is completely fine. It’s not important to move the spheres at the same time. We are working on a new simulation about ratio and proportion where moving two objects at the same time is more important.

We have a custom interaction accessible to keyboard users, but I wanted to reach out about iOS and VoiceOver, specifically, because from our own investigations we don’t think it is possible.

Thoughts from this list are always much appreciated.

Taliesin Smith
talilief@gmail.com<mailto:talilief@gmail.com>

~.~.~
Also reachable at:
Taliesin.Smith@colorado.edu<mailto:Taliesin.Smith@colorado.edu>
Inclusive Design Researcher
PhET Interactive Simulations
https://phet.colorado.edu/en/accessibility

Physics Department
University of Colorado, Boulder