Re: hit testing and retained graphics

On Thu, Jun 30, 2011 at 11:07 PM, Charles Pritchard <chuck@jumis.com> wrote:
> The server running on a remote system can walk the accessibility tree
> and send that information to the remote client, which can then recreate
> the tree using ARIA+HTML in the canvas shadow dom while it manages
> repaints.
>
> It relates only to the remote system use case.

Okay, I think I understand your proposal and I'm 95% certain it's a
non-starter.

Walking the accessibility tree is insufficient for real-world
accessibility on the dominant desktop platform, and translating the
accessibility tree will break accessibility even when client and server
are on the same platform.

The solution I proposed in January would not have these problems.

> You state the following:
> "simply converting the UI Automation accessibility tree to a standard format
> and then pushing it over the network would be fairly
> useless to client applications"
>
> But that's exactly what AT software does. Yes, some software requires
> specialization.
> That's why AT vendors exist, and people pay them money for services.

I doubt there's a viable market for your solution.

My impression is that Windows AT vendors already have too much on their
plate translating Windows applications into their own interface once.

For example, GW-Micro is struggling to revamp even their interfaces to
Firefox and Internet Explorer to support recent developments on the web.

The additional job of translating those Windows applications into DOM
and then recreating their normal interface on top of that DOM would double
the work they have to do.

Orca's positively starved of development resource, and struggles to
support even one web browser. Apple hasn't fully enabled access to its
own office applications.

It seems vanishingly unlikely that a developer of AT on one platform
that has not achieved full coverage of the most common apps on that
platform is going to spend time retrofitting to support other
applications from another platform converted into DOM.

Given your solution wouldn't even work for common Windows applications,
it's truly hopeless.

Equally, I can't see this massive challenge being tackled by remote access
implementors.

The approach I suggested in January does not suffer from the same costs
of bringing it to market, so it seems more likely to result in
accessibility in the real world.

> Additionally, basic pass-through of focus events does have benefits. It's
> not an all-or-nothing situation.

I think you're saying that while the proposed extensions to canvas might
not solve the remote access use-case, they might solve other use-cases?

I'm just arguing Sean should not have offered this remote access use-case as
rationale for those extensions, since they don't solve it.

>> You mention exposing "a11y information" but as I tried to explain
>> in my email in January I don't think this would work at all.
>>
>> But exposing "a11y information" in the case of Windows means
>> exposing basically every API in the system, since in practice AT has to
>> hack around the limitations of MSAA, e.g. using display hooks,
>> mirror drivers, and application-specific APIs. Simply passing through UI
>> Automation calls will not allow a remote JAWS instance (say) to access
>> common Windows applications.
>
> We're exposing the information to the OS API; we're exposing
> very few OS APIs back to the scripting environment.

And this is one reason why your approach won't work.

> We're not telling JAWS that it needs to recognize win apps as windows apps,
> we're just telling the UA that there are UI elements with particular roles,
> and particular screen locations. JAWS, and other items, take it from there,
> thanks to ARIA support and IAccessible2 and other methods.

I struggle to imagine anyone wasting time trying to create such a
terrible user experience, when they could just provide familiar access
to JAWS and Microsoft Word on the remote system.

>> Even aside from the shoddiness of Windows accessibility, to make access
>> system independent you'd need to translate the platform accessibility
>> APIs into a common API such as ARIA. Retranslation back into platform
>> accessibility APIs on the other end would obviously result in an
>> accessibility tree with differences from the tree normally presented by
>
> Agreed. There is work that would have to be done by developers.

Get real. Nobody's going to do that work. Please let's talk about
feasible solutions that don't require so much extra work for so little
gain!

> They can't do that work if they're prohibited by UA vendors in the first
> place.

UA vendors aren't obligated to do work to support the remote access
use-case.

> That's a sore issue for me. I've stated many times, that prohibiting
> canvas a11y authoring borders on anti-competitive behavior.

Yes, I've no sympathy for this line of argument as you know.

>> the application. This would likely break custom code for particular
>> applications, such as Safari's VoiceOver item chooser menu, Orca's
>> Firefox script, or Window-Eyes's Firefox code.
>
> I don't understand, here. The ATs only recognize the UA, they're not
> recognizing the remote application.
>
> The ATs are designed to recognize proper semantic markup in the UA/ARIA;
> Given appropriate markup in the DOM, what's the problem.

The problem is this sort of translation more than doubles the work that
remote access and AT implementors have to do, for a terrible user
experience, for a tiny edge case. So we can be confident it won't happen.

--
Benjamin Hawkes-Lewis

Received on Thursday, 30 June 2011 22:49:32 UTC