Re: hit testing and retained graphics

On Fri, Jul 1, 2011 at 2:42 AM, Charles Pritchard <chuck@jumis.com> wrote:
>> Okay, I think I understand your proposal and I'm 95% certain it's a
>> non-starter.
>
> I'm confused: what is it you're referring to you when you talk about my
> proposal?
>
> This held two proposals:
> http://lists.w3.org/Archives/Public/public-canvas-api/2011AprJun/0047.html
>
> Are you referring to remote clients as a use-case, being a non-starter?

No. I'm saying that constructing a DOM for remote system access, as
opposed to providing streams of visual/audio/braille information, is
impractical. If that's the only way we can approach the use case, we
might as well declare it unsolved in HTML5.

>> Walking the accessibility tree is insufficient for real-world
>> accessibility on the dominant desktop platform, and translating the
>> accessibility tree will break accessibility even when client and server
>> are on the same platform.
>>
>> The solution I proposed in January would not have these problems.
>
> You proposed a braille event stream; something like that can simply
> be accomplished through the aria-live property as it currently exists,

How would you tell the local system that an aria-live stream should be
brailled *only*? It's not a *stylistic* difference, so I don't think CSS
@media would be appropriate, even if AT used the "braille" media type,
which it doesn't. How would you achieve the same sync between audio
output and braille output that you would with remote AT?

> They are good comments on technology and how one might handle sending
> data from remote clients. They are not a full solution.

Probably not, but they were intended as the start of a discussion. :)
What else do we need to solve the remote system access use-case?

> And translating accessibility trees are out of the scope of this
> discussion...

Your approach to the remote system use-case depends on such translation,
so I don't see how it can be out of scope?

> The way you've been bringing them up seems to suggest that it's "not
> possible" to work with remote clients and AT -- and that's just not
> factually accurate.

I argued your approach to remote system access was impractical not
impossible.

It might be actually impossible in so far as the web accessibility stack
might not be able to represent all the custom controls and views for
which remote AT can make an accessible interface. But its
more basic impracticality makes that brick wall irrelevant.

>>> You state the following: "simply converting the UI Automation
>>> accessibility tree to a standard format and then pushing it over the
>>> network would be fairly useless to client applications"
>>>
>>> But that's exactly what AT software does. Yes, some software
>>> requires specialization. That's why AT vendors exist, and people pay
>>> them money for services.

The fact that some people pay for AT to do *more* than simply read the
accessibility tree supports my argument that converting the
accessibility tree is insufficient, and therefore providing remote
system access this way would be impractical.

>> I doubt there's a viable market for your solution.
>
> While you're expressing your doubt, keep in mind, I'm not putting forth an
> investment pitch.

You claimed there was a market; I'm saying there isn't one.

I'm not sure what keeping in mind that you're not making an investment
pitch is supposed to do? Remind me that you're not focusing on the
practicality of your proposal?

>> My impression is that Windows AT vendors already have too much on
>> their plate translating Windows applications into their own interface
>> once.
>>
>> For example, GW-Micro is struggling to revamp even their interfaces
>> to Firefox and Internet Explorer to support recent developments on
>> the web.
>
> Take this one: Window-Eyes and JAWS are expensive. They're expensive
> because they cost a lot to maintain, and the market is relatively
> small. Otherwise, as software packages, they might be less expensive.
> Yes, they have a lot on their plate, competition is small, the market
> is difficult, and corporations can be unhelpful.

These facts support my argument that your proposal is impractical as an
approach to remote system access.

>> The additional job of translating those Windows applications into DOM
>> and then recreating their normal interface on top of that DOM would
>> double the work they have to do.
>
> Perhaps we have a misunderstanding here... I'm not slightly suggesting
> that they would do this work. I'm saying that other vendors (or
> existing ones), would do that kind of work, because they can.

I very much doubt it.

Can you find someone willing to commit, say, to providing remote system
access to the Windows platform to all other platforms by translating a
similar coverage of apps to what (say) Window-Eyes provides into DOM and
providing a plugin system so end-users could extend that coverage as
they can with Window-Eyes?

> If someone wants to walk the accessibility tree for LibreOffice, push
> it over the wire, and call it a product, they don't need to wait on
> vendors to do so...

The use-case being discussed here is remote system access, not remote
app access. I refer you back to Sean's links:

http://www.ericom.com/AccessNow_FAQs.asp

http://www.thinvnc.com/index.html

These applications provide access to the whole system, not individual
apps, and it's a familiar use-case to me, as I've actually used software
like this.

I'm not sure I understand the remote app access use-case. Can you please
link to some real-world examples? I'm interested in both web-based and
native examples.

>> It seems vanishingly unlikely that a developer of AT on one platform
>> that has not achieved full coverage of the most common apps on that
>> platform is going to spend time retrofitting to support other
>> applications from another platform converted into DOM.
>
> It seems to me, utterly absurd, the direction you're taking this with
> me.  It's as if you're saying: existing corporations have not put in
> sufficient resources to maintain their own software; therefore, new
> companies are not going to work on it.

I'm saying that the people (accessibility devs, remote system access
devs) with the skills and inclination to tackle this problem are already
fully engaged. An approach that more than doubles their workload to
support a small edge-case is a non-starter compared to paving the
cowpath of how native accessible remote system access works.

It's striking that even though remote system access desktop software
*could* make use of local accessibility trees to provide access to local
AT, that's *not* how remote system access works today (as I documented
in my January email). Your proposal essentially involves the same thing,
but with layers of (potentially complicating) indirection via the DOM.
So the non-existence of this approach today provides some suggestive
empirical evidence that it won't suddenly appear tomorrow.

> But that's not quite what you're stating here, you are also saying that,
> companies would have to have support for a "full coverage of common" apps,
> and that's just not true.
>
> Simply supporting a remote session of one common office application,
> such as a spreadsheet, is enough to build business upon.

That's not remote system access like AccessNow or ThinVNC, it's a
different class of product and use-case altogether.

>> Equally, I can't see this massive challenge being tackled by remote
>> access implementors.
>
> Well I'm sorry you feel that way about web developers. There are so
> many of them, and some of them very much have an interest in these
> kinds of activities.

Please provide examples for the remote system access use-case.

> I'd prefer that vendors were actually on-board.
>
> This kind of work is required of them, per Accessibility Guidelines;
> it's noted as programmatic access.

No law, standard, specification, or guidance requires user agents to
support uses of <canvas> not sanctioned by the spec, though such
documents probably do require developers, including companies who should
know better such as IBM, not to abuse <canvas> to create inaccessible
UIs.

More importantly, my claim is that "[t]his kind of work" would not
benefit remote system access. Whether there are independent reasons to
undertake "[t]his kind of work" does not make it correct to cite remote
system access as a use-case.

>> The approach I suggested in January does not suffer from the same
>> costs of bringing it to market, so it seems more likely to result in
>> accessibility in the real world.
>
> The approach you're suggesting is something you might take to a
> corporation, such as JAWS, and pitch them with... Or you might take it
> to a client that's asked for that kind of solution.

I don't follow.

> It's not exclusive, nor does it serve, the purposes that we've
> discussed here about filling out the existing accessibility tree with
> information on bounding boxes and paths.

Sean suggested that such features would help remote system access. I'm
saying they wouldn't. Obviously, that doesn't mean those features
might not be useful for something else. It just means remote system
access should not be cited as a use-case for introducing such features.

> It's a fine detail of things that may work in the future... I'm glad
> to have it, but you speak of it as though it excludes the use case
> we've documented.

If by "use case" you mean remote system access, then yes I'm saying my
approach would be practical and yours would be impractical.

Otherwise, my suggested approach for remote system access is not meant
to exclude other use-cases. I was picking out some of Sean's use-cases
and saying they should not be used to justify the proposed features.

>>> Additionally, basic pass-through of focus events does have benefits.
>>> It's not an all-or-nothing situation.
>>
>> I think you're saying that while the proposed extensions to canvas
>> might not solve the remote access use-case, they might solve other
>> use-cases?
>>
>> I'm just arguing Sean should not have offered this remote access
>> use-case as rationale for those extensions, since they don't solve
>> it.
>
> They improve the situation.

Only if developers are likely to make use of these features to provide
accessible remote system access. I'm arguing that history, market logic,
and the shift towards cloud computing strongly imply they won't.

> Remote access is a very large field.

So?

> Enabling the client to know where the cursor is, the size of the UI
> object behind the cursor, and where the active focus is, are critical
> components.

They are critical components in your impractical approach, but not in
my practical approach.

> They do not "solve" anything, they simply allow for more
> usable solutions to be built.

How would remote system access solutions built using your impractical
approach be "more usable" than the solutions built using my practical
approach?

> This is an all-or-nothing kind of statement:
> "Sean should not have offered this remote access use-case  as rationale...
> since [it doesn't] solve it"
>
> Sean can't "solve" remote access with a few sentences. That's not what
> he's trying to do.  He's stating that remote access is a use case
> which would benefit from the a11y tree.

Okay, sorry, "solve" was the wrong choice of word.

Sean should not have offered remote system access as a rationale, since
these API features don't even /benefit/ it.

>>> Agreed. There is work that would have to be done by developers.
>>
>> Get real. Nobody's going to do that work. Please let's talk about
>> feasible solutions that don't require so much extra work for so little
>> gain!
>
> This is the condescending attitude

It's not "condescending" to assume developers are rational actors who want
to make good use of their time.

> This is an issue of personal freedom, for users and for developers.

No. This is an issue of what constraints we shall place on HTML5
compliant user agent developers that we expect to *result* in better
accessibility.

> What people choose to do with their time, who they choose to do it
> for, and what resources they expend are their own personal
> privilege.
>
> Wendy Chisholm has told me about Turri:
> http://en.wikipedia.org/wiki/Pellegrino_Turri
>
> "Italian Pellegrino Turri invented a mechanical typing machine, one of the
> first typewriters, sometime before 1808, for his blind lover Countess
> Carolina Fantoni da Fivizzono."
>
> Are you really going to put bounds on the entirety of the worlds population,
> and what they will and won't spend time on to connect with others?

People are free to build their own user agents.

You are the one proposing constraints on developers (user-agent
developers). I'm saying such constraints should be based on a realistic
model of how developers of all sorts will act when such constraints are
applied. The particular constraints you are advocating might have other
positive effects, but they are very unlikely to result in more
accessible remote system access.

Saying that developers would have the freedom to act differently
does not challenge my model of how they will *likely* act.

>>> They can't do that work if they're prohibited by UA vendors in the
>>> first place.
>>
>> UA vendors aren't obligated to do work to support the remote access
>> use-case.
>
> That's correct; but we're trying to put out measurable, real world
> use-cases, because Tab has asked for them.

But in the case of remote system access the proposed features and
the use-case are mismatched.

> Otherwise we'd just be focused on fixing the issue.

I think that waving remote system access around as a reason to introduce
various sub-DOM-related constraints on user agents betrays any people
with disabilities who want remote system access over the web, because it
won't deliver it in practice.

> UA vendors ARE obligated to follow Accessibility Guidelines.
>
> They have failed to do so, in the case of canvas.

I think there's wide agreement that software developers are morally
obligated to make reasonable accomodations for accessibility, and that
in many jurisdictions governmental and commercial services have legal
obligations around these lines. This does not automatically apply to any
*particular* guidelines.

That developers are ignoring their obligations by abusing canvas to
create inaccessible UIs does not mean UI vendors have not followed
accessibility guidelines like 508 or UAAG by failing to develop features
to retrofit accessibility against such abuse.

Moreover, I don't see what this question of obligations has to do with
whether your approach is a good approach for remote system access.

>>> That's a sore issue for me. I've stated many times, that prohibiting
>>> canvas a11y authoring borders on anti-competitive behavior.
>>
>> Yes, I've no sympathy for this line of argument as you know.
>
> Legal arguments don't need sympathy, just evidence and pressure.

(a) IANAL, but I'm unpersuaded by your legal arguments.

(b) Even if I accepted your "[l]egal arguments", that would not justify
introducing remote system access as a use-case for the proposed
features.

> Stop scoffing at an industry, it's silly for you to take that stance.

It would be helpful if you came up with actual arguments against my
model of what developers will likely do in practice, rather than
characterising my model as "condescending" or "scoffing" or "silly" or
"absurd", since these are value judgements I don't share.

Do you not see that it is counter-productive when these conversations go
round in circles like this:

Person A: We need to help developers make accessible web services.
Person B: Agreed.
Person A: We need feature X to support accessibility use-cases J and K.
Person B: Feature X does not benefit use-case J, but it sounds like a
good use-case to try to solve. How about feature Y?
Person A: That argument is as dumb as bricks. User agents are failing their
moral and legal obligations by not providing accessibility and enabling
competition. We need feature X to support accessibility use-cases J and
K.
Person B: <facepalm>

Imagine another world for a second in which these conversations went
more like this:

Person A: We need to help developers make accessible web services.
Person B: Agreed.
Person A: We need feature X to support accessibility use-cases J and K.
Person B: Feature X does not benefit use-case J, but it sounds like a
good use-case to try to solve. How about feature Y?
Person A: You're right, but in that case we'd also need feature Z.
Person B: Yeah! Let's add features Y and Z to the spec. <smile>
Person A: Cool! Now what about use-case K?

Wouldn't that be better?

--
Benjamin Hawkes-Lewis

Received on Saturday, 2 July 2011 09:09:15 UTC