Re: hit testing and retained graphics

Hi Ben,
just been looking at canvas based http://demo.ericomaccessnow.com/ (requires
websockets enabled browser, works fine in chrome)

In regards to use with screen magnifcation software, The above product does
not provide information about current focus, so screen magnifiers cannot
track focus, do you think there is a solution for that?

regards
Stevef

On 30 June 2011 23:48, Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com>wrote:

> On Thu, Jun 30, 2011 at 11:07 PM, Charles Pritchard <chuck@jumis.com>
> wrote:
> > The server running on a remote system can walk the accessibility tree
> > and send that information to the remote client, which can then recreate
> > the tree using ARIA+HTML in the canvas shadow dom while it manages
> > repaints.
> >
> > It relates only to the remote system use case.
>
> Okay, I think I understand your proposal and I'm 95% certain it's a
> non-starter.
>
> Walking the accessibility tree is insufficient for real-world
> accessibility on the dominant desktop platform, and translating the
> accessibility tree will break accessibility even when client and server
> are on the same platform.
>
> The solution I proposed in January would not have these problems.
>
> > You state the following:
> > "simply converting the UI Automation accessibility tree to a standard
> format
> > and then pushing it over the network would be fairly
> > useless to client applications"
> >
> > But that's exactly what AT software does. Yes, some software requires
> > specialization.
> > That's why AT vendors exist, and people pay them money for services.
>
> I doubt there's a viable market for your solution.
>
> My impression is that Windows AT vendors already have too much on their
> plate translating Windows applications into their own interface once.
>
> For example, GW-Micro is struggling to revamp even their interfaces to
> Firefox and Internet Explorer to support recent developments on the web.
>
> The additional job of translating those Windows applications into DOM
> and then recreating their normal interface on top of that DOM would double
> the work they have to do.
>
> Orca's positively starved of development resource, and struggles to
> support even one web browser. Apple hasn't fully enabled access to its
> own office applications.
>
> It seems vanishingly unlikely that a developer of AT on one platform
> that has not achieved full coverage of the most common apps on that
> platform is going to spend time retrofitting to support other
> applications from another platform converted into DOM.
>
> Given your solution wouldn't even work for common Windows applications,
> it's truly hopeless.
>
> Equally, I can't see this massive challenge being tackled by remote access
> implementors.
>
> The approach I suggested in January does not suffer from the same costs
> of bringing it to market, so it seems more likely to result in
> accessibility in the real world.
>
> > Additionally, basic pass-through of focus events does have benefits. It's
> > not an all-or-nothing situation.
>
> I think you're saying that while the proposed extensions to canvas might
> not solve the remote access use-case, they might solve other use-cases?
>
> I'm just arguing Sean should not have offered this remote access use-case
> as
> rationale for those extensions, since they don't solve it.
>
> >> You mention exposing "a11y information" but as I tried to explain
> >> in my email in January I don't think this would work at all.
> >>
> >> But exposing "a11y information" in the case of Windows means
> >> exposing basically every API in the system, since in practice AT has to
> >> hack around the limitations of MSAA, e.g. using display hooks,
> >> mirror drivers, and application-specific APIs. Simply passing through UI
> >> Automation calls will not allow a remote JAWS instance (say) to access
> >> common Windows applications.
> >
> > We're exposing the information to the OS API; we're exposing
> > very few OS APIs back to the scripting environment.
>
> And this is one reason why your approach won't work.
>
> > We're not telling JAWS that it needs to recognize win apps as windows
> apps,
> > we're just telling the UA that there are UI elements with particular
> roles,
> > and particular screen locations. JAWS, and other items, take it from
> there,
> > thanks to ARIA support and IAccessible2 and other methods.
>
> I struggle to imagine anyone wasting time trying to create such a
> terrible user experience, when they could just provide familiar access
> to JAWS and Microsoft Word on the remote system.
>
> >> Even aside from the shoddiness of Windows accessibility, to make access
> >> system independent you'd need to translate the platform accessibility
> >> APIs into a common API such as ARIA. Retranslation back into platform
> >> accessibility APIs on the other end would obviously result in an
> >> accessibility tree with differences from the tree normally presented by
> >
> > Agreed. There is work that would have to be done by developers.
>
> Get real. Nobody's going to do that work. Please let's talk about
> feasible solutions that don't require so much extra work for so little
> gain!
>
> > They can't do that work if they're prohibited by UA vendors in the first
> > place.
>
> UA vendors aren't obligated to do work to support the remote access
> use-case.
>
> > That's a sore issue for me. I've stated many times, that prohibiting
> > canvas a11y authoring borders on anti-competitive behavior.
>
> Yes, I've no sympathy for this line of argument as you know.
>
> >> the application. This would likely break custom code for particular
> >> applications, such as Safari's VoiceOver item chooser menu, Orca's
> >> Firefox script, or Window-Eyes's Firefox code.
> >
> > I don't understand, here. The ATs only recognize the UA, they're not
> > recognizing the remote application.
> >
> > The ATs are designed to recognize proper semantic markup in the UA/ARIA;
> > Given appropriate markup in the DOM, what's the problem.
>
> The problem is this sort of translation more than doubles the work that
> remote access and AT implementors have to do, for a terrible user
> experience, for a tiny edge case. So we can be confident it won't happen.
>
> --
> Benjamin Hawkes-Lewis
>
>


-- 
with regards

Steve Faulkner
Technical Director - TPG

www.paciellogroup.com | www.HTML5accessibility.com |
www.twitter.com/stevefaulkner
HTML5: Techniques for providing useful text alternatives -
dev.w3.org/html5/alt-techniques/
Web Accessibility Toolbar - www.paciellogroup.com/resources/wat-ie-about.html