Re: You Got Your SVG in my Canvas! Mmm, Delicious! (was: hit testing and retained graphics)

Tab, I am cc'ing Paul Bakaus from Zynga. I don't know if he is monitoring
this list.

Rich Schwerdtfeger
CTO Accessibility Software Group

"Tab Atkins Jr." <jackalmage@gmail.com> wrote on 06/30/2011 11:18:39 AM:

> From: "Tab Atkins Jr." <jackalmage@gmail.com>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc: public-canvas-api@w3.org, public-canvas-api-request@w3.org, Doug
> Schepers <schepers@w3.org>
> Date: 06/30/2011 11:19 AM
> Subject: Re: You Got Your SVG in my Canvas! Mmm, Delicious! (was:
> hit testing and retained graphics)
>
> On Thu, Jun 30, 2011 at 5:49 AM, Richard Schwerdtfeger
> <schwer@us.ibm.com> wrote:
> >
> > Tab,
> >
> > You were explained to more than once what problems we are trying to
solve.
> >
> > 1. We need a vehicle to tell an assistive technology the position
> and bounds of an object on the drawing space.
> > 2. In EVERY operating system since 1994 this has been tied to
> retained mode graphics information. In these systems hit testing was
> tied to the same information in retained mode graphics used to
> supply platform accessibility APIs.
> > 3. Authors have been asking for hit testing
>
> This is why I keep pounding on the "define the problems you're solving
> first" thing.  You keep trying to jump straight to the solution.  Your
> #1 and #3 are good problems to solve.
>

OK. I think I gave you the problem I am trying to solve for accessibility.
EVERY platform accessibility API on EVERY platform provides the bounds of
an accessible object to an AT for the reasons I stated. No additional
problem is being stated here.

Zynga and others want a faster way to do hit testing by the browser. They
essentially have to do their own hit testing. To do hit testing you have to
know where all the objects are. I have also provided to you some of work
arounds authors.

Zynga would like to minimize the number of DOM elements used in the canvas
subtree. This is in conflict with accessibility because we need to have an
accessible object produced from the DOM that represents drawing objects on
canvas. Zynga believes accessibility is important but their priority for
their users is speed. So, the solution may mean to be such that 1 or more
drawing paths are assigned a unique ID and those paths are each wired to
one or more DOM elements that would process the events. The purpose of the
ID would be such that the DOM subtree element can be made aware of what
drawing object received the hit.

> Note that your #2 is suggesting that, based on history, a correct
> solution is likely going to be based on a retained-mode API.  *I 100%
> agree.*  However, without a proper list of problems, there's no way to
> evaluate the possible solutions.  They may not even all be solved by
> the same thing.
>
Well, now you have them and I have added some additional requirements for
Zynga.

> For example, hit-testing is a useful thing to solve.  More importantly,
it's:
>
> 1. Self-evidently useful to the author that needs it
> 2. Immediately apparent when you're doing it wrong
> 3. Completely safe to ignore if you don't need it for your app
>
Now you are providing requirements that make it more acceptable to you.
That is the first time we have heard of these. We are not mind readers and
this dreadfully laborious way of communicating via snail mail is not a lot
of help.

1. Authors needing to support accessibility will be required to use the hit
testing vehicle with an explanation as to why.
2. At least for accessibility, if you are unable to determine the proper
location of the canvas sub tree DOM element associated with the drawing
object on canvas then a magnifier will be unable to zoom to that location.
Also, if you create a path and it is not assigned to a valid DOM element
you will definitely fail. Also, if in hit testing the associated DOM
element is removed or hidden it should also fail.
3. If you don't want to use hit testing on drawing objects don't add them
and canvas will in fact route the hit events off the canvas element like it
does now. If an author never introduces a drawing path that needs to be hit
tested then

> These three conditions make it *much* more acceptable to solve the
> problem by grafting an optional API onto the 2d context.  If the
> author gets it wrong, they *know*, immediately, and it's a bug worth
> spending time fixing.  If the author ignores it because they don't
> need it, that's completely fine, because it *doesn't hurt the user*.
> (It still doesn't mean this is the best solution, of course - "use
> SVG" may still be the right answer, possibly along with "let's fix
> your problems with SVG".  Or "use this new canvas context designed to
> handle hit-testing well".)
>
> Your problem #1, though, meets *none* of those criteria.  It's not
> obvious to the average author that it's important.  If they're doing
> it wrong, they won't find out through normal app use (they'd have to
> explicitly test it with an AT).  And if they ignore it, it hurts the
> user.
>
> The same applies to the problem I extracted from Charles Pritchard's
link.
>
> Because they usually fail these three criteria, accessibility problems
> are simply harder to solve.  You have to smuggle in their solutions so
> that they're solved automatically while the author is using the
> technology normally, solving problems that *do* meet the three
> criteria.  This is not a point up for debate - it is a simple truth
> that, unfortunately, many people in this conversation seem not to
> recognize.  Any solution that ignores this fact will end up a complete
> waste of time that doesn't actually measurably improve accessibility.

I don't see the harder to solve problem. I will say that without ability of
an assistive technology to find the objects and assess their dimensions
breaks interoperability with platform accessibility APIs on all platforms.
Without the information a user agent cannot fulfill its accessibility API
obligations need to drive a magnifier or a screen reader that outputs
Braille.

>
> ~TJ