Re: Focus and Combobox Interaction with Mobile VoiceOver

Thanks Mark,
This might sound weird, but in our implementation, the visual experience is completely drawn on the screen with a custom screen graph. I am only interested in the non-visual experience that is rendered in parallel in HTML and ARIA. The HTML experience is how we make the graphically rendered simulations accessible.

So, I was specifically asking about the gestures that would operate the the fortune wheel-like experience when Voice Over is enabled.

All this is really good information.
Taliesin

> On Jan 28, 2020, at 3:56 AM, Marc Haunschild <haunschild@mhis.onmicrosoft.de> wrote:
> 
> Hi Taliesin,
> 
> Just to make things clear: you want to replace a select by a self-made component, right?
> 
> That means the native behaviour of the native element on iOS is NOT a combo box, that stays open - it’s this thing at the bottom of the screen looking like a fortune wheel, where you can choose options from.
> 
> That's what users are used to, what (most of them) can deal with and it’s the one with the best user experience (although select never is really user friendly)…
> 
> Marc
> 
>> On 28. Jan 2020, at 02:51, Taliesin Smith <talilief@gmail.com <mailto:talilief@gmail.com>> wrote:
>> 
>> Hi Matt,
>> Thanks on the heads up on changes to the combobox role. That’s good to know and when the changes are made, hopefully browsers and AT will support the changes.
>> 
>> To answer your question, no, the box does not stay open in any other case. 
>> 
>> This is only an issue while using VoiceOver touch gestures on iOS with VoiceOver enabled. The list of options automatically collapses with loss of focus in all other cases with or without screen reader on.
>> 
>> The fact that is only an issue with VO gestures on iOS leads me to believe that it might be best to treat the popped-up list of of options like a popped-up dialog on that platform. I will discuss with my team how feasible this is.
>> 
>> From Sina’s comments it seems that this is the native behavior on iOS, so it is worth investigating further.
>> It is also the native behavior of the HTML select element which we wanted to use, but can’t due to visual display limitations.
>> 
>> I really appreciate everyone’s comments.
>> 
>> Thanks,
>> Taliesin
>> 
>> 
>> 
>>> On Jan 27, 2020, at 8:43 PM, Matt King <a11ythinker@gmail.com <mailto:a11ythinker@gmail.com>> wrote:
>>> 
>>> Hi Taliesin,
>>>  
>>> Did you figure out what you would like to do with this?
>>>  
>>> If a screen reader is not running, can users interact with the slider when the listbox is expanded? If so, I assume interacting with the slider would collapse the listbox? The same type of interaction should be OK for screen reader users.
>>>  
>>> That said, as a VoiceOver user, I find those types of interactions more tedious on web than in native mobile. In native mobile, pop-up menus and lists are often modal and that can simplify the interaction … except when it is buggy and you can’t get out of the modal ☹. Attempting to replicate that for mobile web is an interesting idea, but it would be a significant change to the patterns and could be pretty tricky. It could have some undesirable side effects for desktop users.
>>>  
>>> BTW, coming changes to the combobox role in ARIA 1.2 will make this kind of widget simpler and more flexible. We may be rewriting that collapsible listbox pattern using combobox later this year.
>>>  
>>> Matt King
>>>  
>>> From: Taliesin Smith <talilief@gmail.com <mailto:talilief@gmail.com>> 
>>> Sent: Sunday, January 19, 2020 8:07 PM
>>> To: w3c-wai-ig@w3.org <mailto:w3c-wai-ig@w3.org>
>>> Cc: Jesse Greenberg <jesse.greenberg@colorado.edu <mailto:jesse.greenberg@colorado.edu>>
>>> Subject: Focus and Combobox Interaction with Mobile VoiceOver
>>>  
>>> Hi Folks,
>>> I have a question about how strict focus should be kept on a list of options in a combobox interaction. 
>>>  
>>> The PhET Sim Molarity is implemented with interactive description that can be accessed with four screen reader/browser combinations: NVDA & Firefox, JAWS & Chrome, VoiceOver & Safari, and Mobile VO & Safari on iOS. My question is specifically about Mobile Voice on iOS.
>>>  
>>> Link to Molarity: https://phet-dev.colorado.edu/html/molarity/1.6.0-dev.1/phet/molarity_en_phet.html?a11y <https://phet-dev.colorado.edu/html/molarity/1.6.0-dev.1/phet/molarity_en_phet.html?a11y>
>>>  
>>> The combobox interaction for choosing a new solute is based on the the accessible design pattern used here:
>>> https://www.w3.org/TR/wai-aria-practices/examples/listbox/listbox-collapsible.html <https://www.w3.org/TR/wai-aria-practices/examples/listbox/listbox-collapsible.html>
>>>  
>>> The issue that we have found is that when using Mobile VoiceOve it is possible for the list of options to stay open and still move focus over to a slider and interact with the slider.
>>>  
>>> My question is, would it be better to treat the popped-up list of solutes like a dialog and prevent access to the rest of the sim while the list of options is open?
>>>  
>>> Currently, we are using focus, so when the list loses focus it closes, but this doesn’t seem to work with Mobile VoiceOver.
>>>  
>>> Any thoughts?
>>>  
>>> Taliesin Smith
>>>  
>>> ~.~.~
>>> Also available off list at:
>>> Taliesin.Smith@colorado.edu <mailto:Taliesin.Smith@colorado.edu>
>>> Inclusive Design Researcher
>>> PhET Interactive Simulations
>>> https://phet.colorado.edu/en/accessibility <https://phet.colorado.edu/en/accessibility>
>>> Physics Department
>>> University of Colorado, Boulder
>> 
>