Re: hit testing and retained graphics

On 6/30/2011 3:48 PM, Benjamin Hawkes-Lewis wrote:
> On Thu, Jun 30, 2011 at 11:07 PM, Charles Pritchard<chuck@jumis.com>  wrote:
>> The server running on a remote system can walk the accessibility tree
>> and send that information to the remote client, which can then recreate
>> the tree using ARIA+HTML in the canvas shadow dom while it manages
>> repaints.
>>
>> It relates only to the remote system use case.
> Okay, I think I understand your proposal and I'm 95% certain it's a
> non-starter.
I'm confused: what is it you're referring to you when you talk about my 
proposal?

This held two proposals:
http://lists.w3.org/Archives/Public/public-canvas-api/2011AprJun/0047.html

Are you referring to remote clients as a use-case, being a non-starter?

> Walking the accessibility tree is insufficient for real-world
> accessibility on the dominant desktop platform, and translating the
> accessibility tree will break accessibility even when client and server
> are on the same platform.
>
> The solution I proposed in January would not have these problems.
You proposed a braille event stream; something like that can simply
be accomplished through the aria-live property as it currently exists,
and you proposed a sound stream, something that can be handled
by the html audio element.

They are good comments on technology and how one
might handle sending data from remote clients. They are not a full solution.

And translating accessibility trees are out of the scope of this 
discussion...
The way you've been bringing them up seems to suggest that it's "not 
possible"
to work with remote clients and AT -- and that's just not factually 
accurate.


>> You state the following:
>> "simply converting the UI Automation accessibility tree to a standard format
>> and then pushing it over the network would be fairly
>> useless to client applications"
>>
>> But that's exactly what AT software does. Yes, some software requires
>> specialization.
>> That's why AT vendors exist, and people pay them money for services.
> I doubt there's a viable market for your solution.
While you're expressing your doubt, keep in mind, I'm not putting forth 
an investment pitch.

> My impression is that Windows AT vendors already have too much on their
> plate translating Windows applications into their own interface once.
>
> For example, GW-Micro is struggling to revamp even their interfaces to
> Firefox and Internet Explorer to support recent developments on the web.
Take this one: Window-Eyes and JAWS are expensive. They're expensive because
they cost a lot to maintain, and the market is relatively small. 
Otherwise, as software
packages, they might be less expensive. Yes, they have a lot on their plate,
competition is small, the market is difficult, and corporations can be 
unhelpful.

> The additional job of translating those Windows applications into DOM
> and then recreating their normal interface on top of that DOM would double
> the work they have to do.
Perhaps we have a misunderstanding here... I'm not slightly suggesting 
that they would do
this work. I'm saying that other vendors (or existing ones), would do 
that kind of work,
because they can.

If someone wants to walk the accessibility tree for LibreOffice, push it 
over the wire,
and call it a product, they don't need to wait on vendors to do so... At 
least, they won't have to,
if a11y is available to them in the browser... Which it's not; and I'm 
saying, that's an issue
of market principles. Vendors are "dragging their feet" on canvas a11y. 
They're hurting
small business and depriving consumers.

> Orca's positively starved of development resource, and struggles to
> support even one web browser. Apple hasn't fully enabled access to its
> own office applications.
And they're not doing all that well with Safari, either.
> It seems vanishingly unlikely that a developer of AT on one platform
> that has not achieved full coverage of the most common apps on that
> platform is going to spend time retrofitting to support other
> applications from another platform converted into DOM.
It seems to me, utterly absurd, the direction you're taking this with me.
It's as if you're saying: existing corporations have not put in 
sufficient resources to maintain
their own software; therefore, new companies are not going to work on it.

But that's not quite what you're stating here, you are also saying that,
companies would have to have support for a "full coverage of common" apps,
and that's just not true.

Simply supporting a remote session of one common office application,
such as a spreadsheet, is enough to build business upon.

> Equally, I can't see this massive challenge being tackled by remote access
> implementors.
Well I'm sorry you feel that way about web developers.
There are so many of them, and some of them very much have an interest 
in these
kinds of activities. They are, unfortunately, blocked in their ability 
to express themselves
when browser vendors actively push-back against a11y.

I'd prefer that vendors were actually on-board.

This kind of work is required of them, per Accessibility Guidelines;
it's noted as programmatic access.

> The approach I suggested in January does not suffer from the same costs
> of bringing it to market, so it seems more likely to result in
> accessibility in the real world.
The approach you're suggesting is something you might take to a corporation,
such as JAWS, and pitch them with... Or you might take it to a client 
that's asked
for that kind of solution.

It's not exclusive, nor does it serve, the purposes that we've discussed 
here
about filling out the existing accessibility tree with information on 
bounding boxes
and paths.

It's a fine detail of things that may work in the future... I'm glad to 
have it,
but you speak of it as though it excludes the use case we've documented.
>> Additionally, basic pass-through of focus events does have benefits. It's
>> not an all-or-nothing situation.
> I think you're saying that while the proposed extensions to canvas might
> not solve the remote access use-case, they might solve other use-cases?
>
> I'm just arguing Sean should not have offered this remote access use-case as
> rationale for those extensions, since they don't solve it.
They improve the situation. Remote access is a very large field. 
Enabling the client
to know where the cursor is, the size of the UI object behind the 
cursor, and where
the active focus is, are critical components. They do not "solve" 
anything, they simply
allow for more usable solutions to be built.

This is an all-or-nothing kind of statement:
"Sean should not have offered this remote access use-case  as 
rationale... since [it doesn't] solve it"

Sean can't "solve" remote access with a few sentences. That's not what 
he's trying to do.
He's stating that remote access is a use case which would benefit from 
the a11y tree.

>> We're exposing the information to the OS API; we're exposing
>> very few OS APIs back to the scripting environment.
> And this is one reason why your approach won't work.
Why? We have the entire DOM to work with, with the vocabulary of WAI-ARIA
and HTML5. At the very least, it can match the a11y of whatever is currently
accessible from within the web browser.

And that is what we are trying to improve here, the amount of content
that is accessible from within a web browser.
> I struggle to imagine anyone wasting time trying to create such a
> terrible user experience, when they could just provide familiar access
> to JAWS and Microsoft Word on the remote system.
Pity your poor imagination, for it has never seen without eyes. Woe.

>> Agreed. There is work that would have to be done by developers.
> Get real. Nobody's going to do that work. Please let's talk about
> feasible solutions that don't require so much extra work for so little
> gain!
This is the condescending attitude that I snapped at Tab for earlier,
it's the kind of dismissal that really disappointed me when I saw it 
from Ian Hickson,
and it's utterly shameful to have seen it from Mozilla developers.

It hurts, to see this kind of talk put forward by people I have respect 
for, people
who have earned respect.

This is an issue of personal freedom, for users and for developers. What 
people
choose to do with their time, who they choose to do it for, and what 
resources
they expend are their own personal privilege.

Wendy Chisholm has told me about Turri:
http://en.wikipedia.org/wiki/Pellegrino_Turri

"Italian Pellegrino Turri invented a mechanical typing machine, one of 
the first typewriters, sometime before 1808, for his blind lover 
Countess Carolina Fantoni da Fivizzono."

Are you really going to put bounds on the entirety of the worlds population,
and what they will and won't spend time on to connect with others ?


>> They can't do that work if they're prohibited by UA vendors in the first
>> place.
> UA vendors aren't obligated to do work to support the remote access
> use-case.
That's correct; but we're trying to put out measurable, real world 
use-cases,
because Tab has asked for them. Otherwise we'd just be focused on fixing 
the issue.

UA vendors ARE obligated to follow Accessibility Guidelines.
They have failed to do so, in the case of canvas. That was O.K., back in 
'08.
Following the work done in '09 to develop proposals to fix the situation,
it stopped being O.K.

It's OK when RIM, Apple and Google release a mobile phone without
useful a11y support. It's not OK if they continue to do so after two years.

Apple got their support up and running; Google has to an extent, done 
the same.
RIM is actively working on it, on their PlayBook.


>> That's a sore issue for me. I've stated many times, that prohibiting
>> canvas a11y authoring borders on anti-competitive behavior.
> Yes, I've no sympathy for this line of argument as you know.
Legal arguments don't need sympathy, just evidence and pressure.
And often, they settle out of court. In that sense, it's often a [poor]
business decision to neglect a11y.

>>> the application. This would likely break custom code for particular
>>> applications, such as Safari's VoiceOver item chooser menu, Orca's
>>> Firefox script, or Window-Eyes's Firefox code.
>> I don't understand, here. The ATs only recognize the UA, they're not
>> recognizing the remote application.
>>
>> The ATs are designed to recognize proper semantic markup in the UA/ARIA;
>> Given appropriate markup in the DOM, what's the problem.
> The problem is this sort of translation more than doubles the work that
> remote access and AT implementors have to do, for a terrible user
> experience, for a tiny edge case. So we can be confident it won't happen.

I think you're still mis-understanding... Existing AT implementers plug 
into the
browser already. They use ARIA and HTML semantics. That work is already 
done;
they are agnostic as to what apps are actually running in the browser.
One tiny exception is IE9s compatibility mode list.

As for the person working on remote access for an application: that's 
what they're
working on... so I mean.. they're working on it.. that's what they are 
doing with their time.

Stop scoffing at an industry, it's silly for you to take that stance.

-Charles

Received on Friday, 1 July 2011 01:42:59 UTC