Re: Focus and Combobox Interaction with Mobile VoiceOver

Hi Marc,
As you said, this two-part article is excellent - and very recent (Dec 2019). 

We would love to be able to use an HTML select in our implementation, but can’t because there is no way to visually hide the list of options. To make our interactive science simulations accessible we have two parallel experiences - one rendered graphically in a custom scene graph that uses any combination of canvas, svg and webgl, and a visually hidden but fully accessible described interactive experience rendered in semantically rich HTML (and ARIA).

We are going to try to explore making our popped up list behave more like a dialog, hiding access to the rest of the sim’s content while the list of options is open and until a selection is made.

If this doesn’t work, we can revert.

Thanks again everyone for the thoughts and resources.
Taliesin

> On Jan 28, 2020, at 3:03 AM, Marc Haunschild <haunschild@mhis.onmicrosoft.de> wrote:
> 
>  “Select your poison” by Sarah Higley is maybe the best article covering the „remake select boxes“ topic. Not sure, if you know this article already or if it is to general. I hope it is helpful.
> 
> Sarah dives deeply into the topic and 24a11y gives her two articles to explain the underlying problems, the different solutions, how they meet WCAG success criteria and last but not least how they perform when given to real users.
> 
> Marc Haunschild
> mhis.de <http://mhis.de/> - web designed for YOU!
> 
>> On 28. Jan 2020, at 02:51, Taliesin Smith <talilief@gmail.com <mailto:talilief@gmail.com>> wrote:
>> 
>> Hi Matt,
>> Thanks on the heads up on changes to the combobox role. That’s good to know and when the changes are made, hopefully browsers and AT will support the changes.
>> 
>> To answer your question, no, the box does not stay open in any other case. 
>> 
>> This is only an issue while using VoiceOver touch gestures on iOS with VoiceOver enabled. The list of options automatically collapses with loss of focus in all other cases with or without screen reader on.
>> 
>> The fact that is only an issue with VO gestures on iOS leads me to believe that it might be best to treat the popped-up list of of options like a popped-up dialog on that platform. I will discuss with my team how feasible this is.
>> 
>> From Sina’s comments it seems that this is the native behavior on iOS, so it is worth investigating further.
>> It is also the native behavior of the HTML select element which we wanted to use, but can’t due to visual display limitations.
>> 
>> I really appreciate everyone’s comments.
>> 
>> Thanks,
>> Taliesin
>> 
>> 
>> 
>>> On Jan 27, 2020, at 8:43 PM, Matt King <a11ythinker@gmail.com <mailto:a11ythinker@gmail.com>> wrote:
>>> 
>>> Hi Taliesin,
>>>  
>>> Did you figure out what you would like to do with this?
>>>  
>>> If a screen reader is not running, can users interact with the slider when the listbox is expanded? If so, I assume interacting with the slider would collapse the listbox? The same type of interaction should be OK for screen reader users.
>>>  
>>> That said, as a VoiceOver user, I find those types of interactions more tedious on web than in native mobile. In native mobile, pop-up menus and lists are often modal and that can simplify the interaction … except when it is buggy and you can’t get out of the modal ☹. Attempting to replicate that for mobile web is an interesting idea, but it would be a significant change to the patterns and could be pretty tricky. It could have some undesirable side effects for desktop users.
>>>  
>>> BTW, coming changes to the combobox role in ARIA 1.2 will make this kind of widget simpler and more flexible. We may be rewriting that collapsible listbox pattern using combobox later this year.
>>>  
>>> Matt King
>>>  
>>> From: Taliesin Smith <talilief@gmail.com <mailto:talilief@gmail.com>> 
>>> Sent: Sunday, January 19, 2020 8:07 PM
>>> To: w3c-wai-ig@w3.org <mailto:w3c-wai-ig@w3.org>
>>> Cc: Jesse Greenberg <jesse.greenberg@colorado.edu <mailto:jesse.greenberg@colorado.edu>>
>>> Subject: Focus and Combobox Interaction with Mobile VoiceOver
>>>  
>>> Hi Folks,
>>> I have a question about how strict focus should be kept on a list of options in a combobox interaction. 
>>>  
>>> The PhET Sim Molarity is implemented with interactive description that can be accessed with four screen reader/browser combinations: NVDA & Firefox, JAWS & Chrome, VoiceOver & Safari, and Mobile VO & Safari on iOS. My question is specifically about Mobile Voice on iOS.
>>>  
>>> Link to Molarity: https://phet-dev.colorado.edu/html/molarity/1.6.0-dev.1/phet/molarity_en_phet.html?a11y <https://phet-dev.colorado.edu/html/molarity/1.6.0-dev.1/phet/molarity_en_phet.html?a11y>
>>>  
>>> The combobox interaction for choosing a new solute is based on the the accessible design pattern used here:
>>> https://www.w3.org/TR/wai-aria-practices/examples/listbox/listbox-collapsible.html <https://www.w3.org/TR/wai-aria-practices/examples/listbox/listbox-collapsible.html>
>>>  
>>> The issue that we have found is that when using Mobile VoiceOve it is possible for the list of options to stay open and still move focus over to a slider and interact with the slider.
>>>  
>>> My question is, would it be better to treat the popped-up list of solutes like a dialog and prevent access to the rest of the sim while the list of options is open?
>>>  
>>> Currently, we are using focus, so when the list loses focus it closes, but this doesn’t seem to work with Mobile VoiceOver.
>>>  
>>> Any thoughts?
>>>  
>>> Taliesin Smith
>>>  
>>> ~.~.~
>>> Also available off list at:
>>> Taliesin.Smith@colorado.edu <mailto:Taliesin.Smith@colorado.edu>
>>> Inclusive Design Researcher
>>> PhET Interactive Simulations
>>> https://phet.colorado.edu/en/accessibility <https://phet.colorado.edu/en/accessibility>
>>> Physics Department
>>> University of Colorado, Boulder
>> 
>