Re: draft of hit testing on active regions (actually paths) in canvas

On 10/27/11 5:42 PM, Jonas Sicking wrote:
> Some feedback:
>
> Limiting to only allowing paths seems like a unfortunate limitation.
> For example it misses calls to drawImage which I think is quite
> common. I'd rather prefer a call where you say "all drawing operations
> from this point should be considered drawing for element X", then let
> the page do arbitrary drawing operations, then have a second call
> which says "i'm now drawing for element Y" or "I'm now drawing for no
> element".

These are two very different ideas. I did not realize -how- different 
your proposal was from the one the WG discussed earlier in the year. I'm 
going to try to explain both of them, and please correct me if I've made 
a mistake.

The setDrawingFor proposal monitors subsequent drawing calls and updates 
the bounding box associated with the target element as drawing calls happen.

// start example
ctx.setDrawingFor(element);
ctx.fillRect(0,0,20,20);
ctx.fillRect(200,200, 100,100);
ctx.setDrawingFor(null);
// at end element now has a bounding region from 0,0 to 300,300.

The setPathForElement proposal acts on the current path:
// start example
ctx.fillRect(0,0,20,20);
ctx.fillRect(200,200,100,100);
ctx.setPathForElement(element);
// at end element now has a bounding region from 200,200 to 300,300.

Both of these proposals can be implemented with minimal effort in 
existing Canvas code bases.

I'm concerned about supporting non-rectangular regions in large areas.
http://www.w3.org/Graphics/SVG/WG/track/issues/2421

I'm concerned that the setDrawingFor proposal requires that authors run 
fill or stroke to operate. This would require additional steps for 
authors setting an interactive region:

// Using setDrawingFor without painting
ctx.save();
ctx.setDrawingFor(element);
ctx.fillStyle = 'rgba(0,0,0,0)';
ctx.fillRect(0,0,20,20);
ctx.setDrawingFor(null);
ctx.restore();

// Using setPathForElement without painting
ctx.beginPath();
ctx.rect(0,0,20,20);
ctx.setPathForElement(element);


> What is the use case for the zIndex argument? The actual pixel drawing
> operations hasn't had a need for that so far and instead rely on the
> painters algorithm. It seems better to me to have a direct mapping
> between the drawing operations and the accessibility API.

It may be unnecessary. I'll do some deep thinking.
As you said, authors can typically manage these things.

They can also fall back on multiple layers with CSS+transitions if 
they're getting into animation territory.

The issue came up awhile back. I'll let you know if I have any Aha! 
moments to justify the zIndex argument.
I believe it was intended for performance.

> Why return false rather than throw an exception if the element doesn't
> exist? Also what do you mean by "doesn't exist". If the element
> doesn't exist then how could the page have a reference to it and pass
> that reference to the function? Do you mean "element isn't inside the
> canvas"?

I think this was just borrowed from the drawFocusRing semantics. It does 
seem like throwing an error is more appropriate and useful for authors. 
Yes, the concept is that an element must be inside the canvas.

> In general I think I prefer the API I proposed in [1]. Can you
> describe what problems you were trying to solve with your changes
> compared to that proposal?
>
> [1] http://lists.w3.org/Archives/Public/public-canvas-api/2011JulSep/0195.html

My primary concern here is path fidelity. Apart from the extra steps I 
outlined earlier, would a mature setDrawingFor implementation keep all 
of the paths for accessibility?

setPathForElement instructs the implementation to maintain the already 
constructed path on some kind of stack or other ordered object.

setDrawingFor would -possibly- maintain a series of paths opposed to a 
series of subpaths.

Basic implementations are most likely to just keep bounding box 
information in either case. More advanced implementations will maintain 
some kind of additional information about the interactive region.

These are two very different methods, I like them both. 
setPathForElement is less expensive to implement; less lines of code 
changed. setDrawingFor could trap -more- information, relating more to 
SVG interop than to a11y.


An example of "advanced" implementations:

setPathForElement would simply use the current path, add it to a stack, 
and that stack would be accessible by the UA accessibility APIs. It's 
feasible that the path would be serialized and sent to a supporting AT. 
Otherwise, it'd simply be used for its bounding box information. This 
requires no changes in existing Canvas methods, only the addition of new 
methods.

setDrawingFor would require hooks to be added to most drawing methods. 
It may require additional logic for strokeWidth. An advanced 
implementation may collect drawing calls while setDrawingFor is running, 
serializing them into SVG and adding them into a Component DOM. This 
would be a lot of extra work on the CPU. At this point super-computers 
fit in our pockets. It'd be reasonable that toDataURL('image/svg+xml') 
would return a scene-graph from the Component DOM.

---

Jonas,

Do let me know if I've mis-interpreted anything. My focus is on filling 
out the existing accessibility API with low level commands that I can 
easilly get introduced into existing code bases. The setPathForElement 
proposal is minimally invasive for both web devs and implementers. It's 
simple, graceful and easy to express.

I do believe setDrawingFor has merit but I think it is much heavier to 
implement. It seems like it would work quite well for the component 
model and SVG interoperability.

The two methods do not conflict with each other. I'd like to see 
setPathForElement implemented first, and setDrawingFor experimented with 
in the longer term. setPathForElement solves our immediate issues with 
A11y and closes the book on the a11y problems lodged for this WG. 
setDrawingFor is a great idea for connecting with the Component model as 
it matures.


-Charles


-Charles

Received on Friday, 28 October 2011 02:28:21 UTC