Re: You Got Your SVG in my Canvas! Mmm, Delicious! (was: hit testing and retained graphics)

On Thu, Jun 30, 2011 at 9:59 AM, Richard Schwerdtfeger
<schwer@us.ibm.com> wrote:
> "Tab Atkins Jr." <jackalmage@gmail.com> wrote on 06/30/2011 11:18:39 AM:
>> From: "Tab Atkins Jr." <jackalmage@gmail.com>
>> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
>> Cc: public-canvas-api@w3.org, public-canvas-api-request@w3.org, Doug
>> Schepers <schepers@w3.org>
>> Date: 06/30/2011 11:19 AM
>> Subject: Re: You Got Your SVG in my Canvas! Mmm, Delicious! (was:
>> hit testing and retained graphics)
>>
>> On Thu, Jun 30, 2011 at 5:49 AM, Richard Schwerdtfeger
>> <schwer@us.ibm.com> wrote:
>> >
>> > Tab,
>> >
>> > You were explained to more than once what problems we are trying to
>> > solve.
>> >
>> > 1. We need a vehicle to tell an assistive technology the position
>> and bounds of an object on the drawing space.
>> > 2. In EVERY operating system since 1994 this has been tied to
>> retained mode graphics information. In these systems hit testing was
>> tied to the same information in retained mode graphics used to
>> supply platform accessibility APIs.
>> > 3. Authors have been asking for hit testing
>>
>> This is why I keep pounding on the "define the problems you're solving
>> first" thing. You keep trying to jump straight to the solution. Your
>> #1 and #3 are good problems to solve.
>>
>
> OK. I think I gave you the problem I am trying to solve for accessibility.
> EVERY platform accessibility API on EVERY platform provides the bounds of an
> accessible object to an AT for the reasons I stated. No additional problem
> is being stated here.
>
> Zynga and others want a faster way to do hit testing by the browser. They
> essentially have to do their own hit testing. To do hit testing you have to
> know where all the objects are. I have also provided to you some of work
> arounds authors.
>
> Zynga would like to minimize the number of DOM elements used in the canvas
> subtree. This is in conflict with accessibility because we need to have an
> accessible object produced from the DOM that represents drawing objects on
> canvas. Zynga believes accessibility is important but their priority for
> their users is speed. So, the solution may mean to be such that 1 or more
> drawing paths are assigned a unique ID and those paths are each wired to one
> or more DOM elements that would process the events. The purpose of the ID
> would be such that the DOM subtree element can be made aware of what drawing
> object received the hit.

Again, define your problems.  Making your average game, such as the
ones produced by Zynga, accessible to the blind, for example, can
*not* be accomplished by exposing an alternate subtree.  It can
theoretically be done, but only be exposing completely different
interaction modes, which are most likely fairly tightly coupled with
the design of the game itself.  Different disabled subgroups require
different interaction modes.

There *are* accessibility problems that can be solved by exposing an
alternate subtree (or possibly by other solutions).  Until you list
those problems, we have no way to tell how good your proposed solution
is.


>> Note that your #2 is suggesting that, based on history, a correct
>> solution is likely going to be based on a retained-mode API. *I 100%
>> agree.* However, without a proper list of problems, there's no way to
>> evaluate the possible solutions. They may not even all be solved by
>> the same thing.
>>
> Well, now you have them and I have added some additional requirements for
> Zynga.
>
>> For example, hit-testing is a useful thing to solve. More importantly,
>> it's:
>>
>> 1. Self-evidently useful to the author that needs it
>> 2. Immediately apparent when you're doing it wrong
>> 3. Completely safe to ignore if you don't need it for your app
>>
> Now you are providing requirements that make it more acceptable to you. That
> is the first time we have heard of these. We are not mind readers and this
> dreadfully laborious way of communicating via snail mail is not a lot of
> help.

O_o

If these aren't already an intrinsic part of your thought process,
something is very wrong.  These are pretty basic API design
principles.


> 1. Authors needing to support accessibility will be required to use the hit
> testing vehicle with an explanation as to why.

The vast majority of authors don't need to support accessibility.
They find it perfectly acceptable to exclude the relatively small
fraction of people that can't use their app, as they can then instead
spend their effort on making the app better for the much larger group
of people that are fine with the interaction model.  Thus, this does
not satisfy #1.


> 2. At least for accessibility, if you are unable to determine the proper
> location of the canvas sub tree DOM element associated with the drawing
> object on canvas then a magnifier will be unable to zoom to that location.
> Also, if you create a path and it is not assigned to a valid DOM element you
> will definitely fail. Also, if in hit testing the associated DOM element is
> removed or hidden it should also fail.

The vast majority of authors will never use a magnifier.  If that's
the only way for them to tell they've done it wrong, this does not
satisfy #2.


> 3. If you don't want to use hit testing on drawing objects don't add them
> and canvas will in fact route the hit events off the canvas element like it
> does now. If an author never introduces a drawing path that needs to be hit
> tested then

If you do hit-testing yourself by listening for clicks on the
<canvas>, users can't use a magnifier to zoom into the active area
(something you implied was a problem to be solved in the previous
point).  Thus, this does not satisfy #3,


> I don't see the harder to solve problem. I will say that without ability of
> an assistive technology to find the objects and assess their dimensions
> breaks interoperability with platform accessibility APIs on all platforms.
> Without the information a user agent cannot fulfill its accessibility API
> obligations need to drive a magnifier or a screen reader that outputs
> Braille.

I agree.  That does not automatically imply that the correct solution
is a minimally-invasive grafting of retained-mode features onto the
canvas 2d context.

~TJ

Received on Thursday, 30 June 2011 17:54:21 UTC