Re: hit testing and retained graphics

On 6/30/2011 2:45 PM, Benjamin Hawkes-Lewis wrote:
> On Thu, Jun 30, 2011 at 9:02 PM, Charles Pritchard<chuck@jumis.com>  wrote:
>> On 6/30/2011 10:36 AM, Benjamin Hawkes-Lewis wrote:
> [snip]
>
>>> AFAICT the canvas accessibility APIs proposed would not help the
>>> remote system access use-case.
>>>
>>> I tried to discuss this previously:
>>>
>>> http://lists.w3.org/Archives/Public/public-html-a11y/2011Jan/0164.html
>>>
>>> If this is incorrect could someone explain *how* they would help?
>>>
....
>> load a11y information for the remote application environment, and that
>> would then appear on the local environment without loss of fidelity.
>>
>> Does that get close to an explanation?  These VNC viewers already use
>> a server on the remote desktop, to handle transition into WebSockets
>> -- if the client (the browser supported it), patching those services
>> to include a pass-thru for A11y, would benefit users.
> I don't really follow your proposed solution, or how it relates to
> the subtree APIs being proposed elsewhere on this list.
The server running on a remote system can walk the accessibility tree
and send that information to the remote client, which can then recreate
the tree using ARIA+HTML in the canvas shadow dom while it manages
repaints.

It relates only to the remote system use case.

You state the following:
"simply converting the UI Automation accessibility tree to a standard 
format and then pushing it over the network would be fairly
useless to client applications"

But that's exactly what AT software does. Yes, some software requires 
specialization.
That's why AT vendors exist, and people pay them money for services.

Additionally, basic pass-through of focus events does have benefits. 
It's not
an all-or-nothing situation.

> You mention exposing "a11y information" but as I tried to explain
> in my email in January I don't think this would work at all.
>
> But exposing "a11y information" in the case of Windows means
> exposing basically every API in the system, since in practice AT has to
> hack around the limitations of MSAA, e.g. using display hooks,
> mirror drivers, and application-specific APIs. Simply passing through UI
> Automation calls will not allow a remote JAWS instance (say) to access
> common Windows applications.
We're exposing the information to the OS API; we're exposing
very few OS APIs back to the scripting environment.

We're not telling JAWS that it needs to recognize win apps as windows apps,
we're just telling the UA that there are UI elements with particular roles,
and particular screen locations. JAWS, and other items, take it from there,
thanks to ARIA support and IAccessible2 and other methods.

> Even aside from the shoddiness of Windows accessibility, to make access
> system independent you'd need to translate the platform accessibility
> APIs into a common API such as ARIA. Retranslation back into platform
> accessibility APIs on the other end would obviously result in an
> accessibility tree with differences from the tree normally presented by
Agreed. There is work that would have to be done by developers.
They can't do that work if they're prohibited by UA vendors in the first 
place.

That's a sore issue for me. I've stated many times, that prohibiting 
canvas a11y
authoring borders on anti-competitive behavior.

> the application. This would likely break custom code for particular
> applications, such as Safari's VoiceOver item chooser menu, Orca's
> Firefox script, or Window-Eyes's Firefox code.
I don't understand, here. The ATs only recognize the UA, they're not
recognizing the remote application.

The ATs are designed to recognize proper semantic markup in the UA/ARIA;
Given appropriate markup in the DOM, what's the problem.

-Charles

Received on Thursday, 30 June 2011 22:08:37 UTC