Re: hit testing and retained graphics

On 7/7/2011 9:51 AM, Doug Schepers wrote:
>
>> Unfortunately many browser vendors have been underserved by their
>> teams and/motivations: they are pushing back against supporting a11y
>> for canvas content.
>
> Please stop using rhetoric like this.  It is incorrect and perhaps 
> even disingenuous.  It makes people defensive and is a barrier to 
> reasoned discussion.
>
> Browser vendors have not pushed back on "supporting a11y for canvas 
> content", they have pushed back against adding a retained-mode 
> component to the Canvas 2D API, for obvious reasons: the Canvas 2D API 
> is intended for *immediate-mode* graphics, and we already have a 2D 
> graphics standard for the Open Web Platform: SVG.

They are effectively pushing back against supporting their own UA 
accessibility apis and
the following that, the OS level platform accessibility apis that AT 
software accesses.

Many people who work with AT software have commented on this list, some 
more
rhetorically charged than others, about the need for canvas accessibility.

Vendors do not seem to have a working distinction between accessibility 
APIs and DOM apis.

They have asked implementers who worked on Canvas and/or SVG their 
opinion, but those APIs
and developers are not consumers of this data.

I've put forward an argument: we need to have setElementPath(element) to 
send getBounds from Canvas
through CSS and over to the accessibility tree so it can support 
getBounds by assistive technology.

It's a very simple, straightforward request, passing data structures 
from Canvas 2d over to IAccessible2/
whatever underlying content model is used. Canvas 2d does not retain any 
information.

I call this pushing-back, as vendors seem to be claiming that only SVG 
should or will be supported.
This is not a difficult technical task, the push back is only about 
whether or not it should be "allowed"
in the first place. That's a weird response.

I can't invest in patches to open source software when the drivers of 
those code bases
are telling me that the patch will be rejected on the same principle. 
And these patches are intended
for accessibility.

All accessibility APIs have a method to run getBounds (element) 
regardless of whether
or not the element is in focus. Currently, the shadow dom does not 
support that method.
That should be changed.

Defending the decision to cripple the accessibility API -intentionally-, 
that's a hard thing to do,
but that's exactly what's happened on the thread. And it does get heated.
>
>> Vendors of authoring tools are not even able to
>> expose the accessibility information that might otherwise be shared.
>> Many browser vendors are indeed discouraging it with a vague promise
>> of future specs.
>
> Again, this seems to be rhetoric.  I haven't heard any browser vendor 
> offering any future work in this area.  I have proposed what seems to 
> me like a reasonable solution, which is an API that allows the two 2D 
> graphics APIs (Canvas and SVG) available in all major browsers to be 
> used together.  You have claimed this is "overkill", but have not 
> stated why.
I was greatly heartened to see David Singer of Apple write a reasoned, 
clear and short e-mail about this discussion:
http://lists.w3.org/Archives/Public/public-canvas-api/2011JulSep/0179.html

On this list, several vendors have stated that if there are 
accessibility concerns with SVG, then they are on board with addressing 
them,
they are on board with writing future specs for SVG accessibility. Those 
are promises of future specs outside of canvas.

The list has seen dozens of Canvas apps which are not using SVG. They're 
not using it because it's not necessary.
Using SVG mixed with Canvas is not necessary to enable the methods 
currently only available through calls to drawFocusRing.

Stashing paths in the DOM, to be accessed by the scripting environment 
is not necessary to enable the accessibility,
and it further requires string serialization and deserialization. It's a 
heavy solution. It's requiring a public retained-mode API
when we don't need one to achieve our goals. Yes, it's nice, but it's 
not necessary. It's overkill.

>
> I find it frustrating that we have a point on which the browsers seem 
> to agree (that SVG is the retained-mode graphics option), but rather 
> than building on that foundation, you persist in pursuing a redundant 
> approach that the browser vendors have already rejected.  Is this NIH 
> syndrome on your part, or do you have some specific technical reason 
> for rejecting my proposal?  If so, please state your objection 
> clearly, without hand-waving.
The cost of implementation for other vendors is high, and your proposal 
is incomplete, from a technical end. Also, I'm not looking for 
retained-mode graphics,
I'm just trying to support the a11y api methods that assistive 
technology products use. You've told me to just use a completely 
different API.

It's my firm belief that vendors have simply misunderstood the problem.

They have approved of the canvas shadow dom, and of drawFocusRing. As 
far as technical problems go, there
is little difference between setElementPath and drawFocusRing. Vendors 
have not expressed a working knowledge
of the accessibility API I'm targeting nor of the similarity to 
drawFocusRing in implementation.

I'm happy to review, to help on and to comment on methods of working 
with SVG and Canvas together.
That's not exclusive of my commitments to address canvas a11y issues. I 
feel your proposal is not relevant
to the defect I'm addressing. It is certainly a proposal I can support, 
and am willing to work with, but it's
a completely separate issue.

> I have serious reservations *from an accessibility point of view* 
> about extending the Canvas 2D API toward retained-mode or subtree 
> logical structures further than it has already been pushed.  To wit:
>
> * Issues that have been solved in SVG around the scaled viewport, 
> around rich object structure, around smooth scaling (for crisp 
> displays at all sizes), around shapes which move and change size or 
> shape, and so on, are still problems that accessibility in the Canvas 
> 2D API has yet to pose, much less to solve.
I've worked with scaled view-ports, rich object structures, and smooth 
scaling in canvas. I don't know what you're referring to about "shapes 
which move".
But I'd imagine that requestAnimationFrame was the solution to that one. 
If you would like to know about methods, I'm happy to share them.

That said, your statement is factually inaccurate, and I'm happy to 
provide evidence to show that those problems have been posed and solved.

> * Your proposed solution of adding an element to the canvas subtree 
> which carries shape, size, and location information seems to be 
> simple, but only because you haven't carried it to its logical 
> extreme, which will ultimately be to reinvent SVG (perhaps slightly 
> differently, so that it's not even compatible).  Where does it stop?  
> When is accessibility potential of Canvas complete, and not only 
> available, but either encouraged or mandatory, with your approach?
At present, this is where it stops. We've worked through about seven 
issues; off the top of my head we've solved text resolution (crisp 
display), text positioning, caret position, focus and ARIA.
Some of these issues I discovered by implementing HTML+CSS line boxes.

What is the logical extreme you are talking about? This work is based on 
actual practice. I understand you're making a slippery slope argument...

I've been working for years with canvas and I work with several large 
code bases. These are the issues we've discovered in that time,
implementing HTML forms, RTE, SVG and large (over 10k loc) web apps.

And that is why I feel that I have carried it out to its logical extreme 
and am confident that this is an important solution.
It is a simple solution... your concern seems to be that there may be 
"more" issues. But why should that stop us from
fixing a known issue with a simple solution?
>
> * Your approach either either requires the user to maintain the 
> position it themselves, in which case Cameron's aria-region suggestion 
> [1] seems better and is already reasonably defined as a starting 
> point, or would be done automatically, in which case SVG would be 
> better and is already defined (though admittedly, how it works 
> precisely with SVG would also need to be defined, implemented, and 
> tested).
Yes, the author has to maintain the position themselves. It's only the 
author that knows where
the element has been painted on the screen and when it has been 
re-painted. I've explored using
aria-region semantics within the shadow dom. It requires a lot more work 
of authors and more dom
mutations than does the setElementPath proposal.

What qualifications are you using to compare setElementPath with 
aria-region?
More often then not I'm happy to see multiple solutions implemented. It 
gives me
more flexibility as a developer to address situations I run into.

I did suggest stringify methods for Canvas so that aria-region could 
more easily be supported.

-Charles

Received on Wednesday, 13 July 2011 08:03:11 UTC