Re: hit testing and retained graphics

On Thu, Jun 30, 2011 at 9:02 PM, Charles Pritchard <chuck@jumis.com> wrote:
> On 6/30/2011 10:36 AM, Benjamin Hawkes-Lewis wrote:

[snip]

>> AFAICT the canvas accessibility APIs proposed would not help the
>> remote system access use-case.
>>
>> I tried to discuss this previously:
>>
>> http://lists.w3.org/Archives/Public/public-html-a11y/2011Jan/0164.html
>>
>> If this is incorrect could someone explain *how* they would help?
>>
>
> Many remote solutions (VNC-based, mostly) are being looked at for
> updates, based on the idea that they can now be accessed directly from
> the web.
>
> This is certainly an example that could be picked up, and tied into
> MSAA/UIA/IAccessible2 objects:
> https://github.com/kanaka/noVNC
>
> In doing so, the package would gain a competitive advantage... On
> browsers which have mature for support ARIA, which IE9 and FF4 do
> [limited support in WebKit], the session would have an opportunity to
> load a11y information for the remote application environment, and that
> would then appear on the local environment without loss of fidelity.
>
> Does that get close to an explanation?  These VNC viewers already use
> a server on the remote desktop, to handle transition into WebSockets
> -- if the client (the browser supported it), patching those services
> to include a pass-thru for A11y, would benefit users.

I don't really follow your proposed solution, or how it relates to
the subtree APIs being proposed elsewhere on this list.

You mention exposing "a11y information" but as I tried to explain
in my email in January I don't think this would work at all.

But exposing "a11y information" in the case of Windows means
exposing basically every API in the system, since in practice AT has to
hack around the limitations of MSAA, e.g. using display hooks,
mirror drivers, and application-specific APIs. Simply passing through UI
Automation calls will not allow a remote JAWS instance (say) to access
common Windows applications.

Even aside from the shoddiness of Windows accessibility, to make access
system independent you'd need to translate the platform accessibility
APIs into a common API such as ARIA. Retranslation back into platform
accessibility APIs on the other end would obviously result in an
accessibility tree with differences from the tree normally presented by
the application. This would likely break custom code for particular
applications, such as Safari's VoiceOver item chooser menu, Orca's
Firefox script, or Window-Eyes's Firefox code.

How you actually transmit bytes back and forth, whether using web
sockets or something else, seems irrelevant compared to these more
fundamental problems.

--
Benjamin Hawkes-Lewis

Received on Thursday, 30 June 2011 21:46:10 UTC