Re: hit testing and retained graphics

On 7/7/2011 10:07 AM, Doug Schepers wrote:
> Hi, Steve-
>
> Steve Faulkner wrote (on 7/3/11 8:16 AM):
>>
>> For example if I increase the magnification in my browser to 200%, 
>> when an
>> element or object receives focus is offscreen, the view changes so it 
>> is no
>> longer offscreen. This does not happen if the objects are represented 
>> on the
>> canvas, regardless of whether they are local or remote.
>
> That's because "there is no there there".  In a purely graphical 
> display, as with the immediate-mode of the Canvas 2D API, there's no 
> demarcated area for the magnifier to focus on.  There are several 
> solutions to this:
>
> 1) Make AT that is is "smart" enough to identify an important shape, 
> zoom in on it, and track it if it is moving (this seems like a hard 
> problem, but wouldn't need any changes to specs);
As rendering engines push more into the graphics card, ATs have less 
opportunity to apply direct heuristics to the screen.
Also, these are computing devices; forcing vision recognition for 
virtual components is a last resort. It's slow, too.
>
> 2) Make the shape, size, and position information available through a 
> special accessibility API, updated and maintained in parallel to 
> changes to the rendered shape;
This API already exists (with shape as an exception); it's the 
accessibility tree and its implemented by most vendors as a separate
tree which is updated when changes are made to the DOM tree. It works in 
conjunction with the OS a11y API,
called UI Automation in OS X and Windows (though different things); 
called MSAA in older versions of Windows.

http://www.mozilla.org/access/architecture#MSAA_tree_vs._DOM_tree

>
> 3) Make the shape, size, and position information available through a 
> standard API attached to a DOM element; some of the existing proposals 
> on what this element could be include:
>
> 3a) an imagemap (HTML5 <map> element with <area shape="circle | 
> rectangle | polygon">), updated and maintained by the author in 
> parallel to changes to the rendered shape;
Rejected:
http://www.w3.org/html/wg/tracker/issues/105
>
> 3b) a canvas subtree element (perhaps a <div>) with @aria-region, 
> updated and maintained by the author in parallel to changes to the 
> rendered shape (Cameron's proposal);
Also proposed years ago, and brought to the SVG wg:
http://www.w3.org/html/wg/wiki/ChangeProposals/Map4NotAdom

Unfortunately, this does create some overhead, a lot of extra work in 
the DOM.
The SVG anim* prefix in DOM accessors makes a little more sense than
pushing the data directly into DOM attributes.

There are speed issues here, efficiency issues.
>
> 3c) a canvas subtree element with some other additional semantics, 
> updated and maintained by the author in parallel to changes to the 
> rendered shape, or perhaps automatically updated and maintained via 
> the same API that the author uses to change the rendered shape 
> (Charles and Rich's proposal;
We've suggested a single method which would update the accessibility tree;
we've also suggested using this method to update the hit testing of the UA.
> 3d) an SVG element, automatically updated and maintained via the same 
> API that the author uses to change the rendered shape (Doug's proposal).
I've looked at this as well as aria-region:
http://lists.w3.org/Archives/Public/www-svg/2011Apr/0048.html
https://lists.webkit.org/pipermail/webkit-dev/2011-March/016264.html

>
> I've already stated [1] why I think (3c) is seriously flawed as a 
> general approach, in terms of scope, suitability (i.e. "does it solve 
> the underlying problem"), and amenability to the majority of 
> implementers.  Obviously, I think my proposal has the most potential.
3c again seems to be "flawed" in that it reminds many of retained mode 
semantics.
Are there flaws in the technical implementation: that is, are there 
reasons why vendors would not be
able to easily support this given their limited budgets? Are there 
reasons why AT vendors would not
be able to use it? Are there reasons why developers would not be able to 
use it?

Are there reasons why it would be difficult to spec?

Aren't those reasonable items to look at for suitability?

As for "scope"; we have this confusion with SVG, which we seem to be 
stuck on.
I'm sorry that's the case.

We're really just looking for an improvement on image maps which
works easily with the canvas api; that's all we're up to over here.

-Charles

Received on Thursday, 7 July 2011 17:38:25 UTC