Re: hit testing and retained graphics

Rich Schwerdtfeger
CTO Accessibility Software Group

Cameron McCormack <cam@mcc.id.au> wrote on 06/22/2011 12:20:06 AM:

> From: Cameron McCormack <cam@mcc.id.au>
> To: Charles Pritchard <chuck@jumis.com>
> Cc: Richard Schwerdtfeger/Austin/IBM@IBMUS, Frank Olivier
> <Frank.Olivier@microsoft.com>, Cynthia Shelly <cyns@microsoft.com>,
> "david.bolter@gmail.com" <david.bolter@gmail.com>, "Mike@w3.org"
> <Mike@w3.org>, "public-canvas-api@w3.org" <public-canvas-
> api@w3.org>, "public-html@w3.org" <public-html@w3.org>, "public-
> html-a11y@w3.org" <public-html-a11y@w3.org>
> Date: 06/22/2011 12:20 AM
> Subject: Re: hit testing and retained graphics
>
> [BTW, including public-html-request in the list of recipients should
> just used if you want to subscribe/unsubscribe or if you want to
> contact the list maintainer: http://www.w3.org/Mail/Request]
>
> Just a couple of things, without being on top of the context of this
> proposal.
>
> Charles Pritchard:
> > That last excerpt applies to a few rendering APIs, not just Apple's
Cocoa.
> > There's precedent in each of these APIs for a Path data object.
> > WebGL uses path data by typed arrays. It does not include curves.
> > SVG has normalized path types:
> > http://msdn.microsoft.com/en-us/library/ff971968(v=vs.85).aspx
>
> Normalisation of SVG path data doesn’t seem relevant.  (I think you are
> just pointing out that SVG has has DOM objects that represent path
> segment lists.)
>
> Note that in SVG you cannot create independent SVGAnimatedPathSegData/
> SVGPathSegData objects.  They only come attached to a <path> element.
>
> > I do not think we need "retained graphics" in canvas -- but it'd be
> > a huge boost to developers,
> > and to accessibility, if we can have a means for sharing path data
> > between layers.
>
> If you want to be able to store a path to re-use later, what about using
> SVG path data strings?  That doesn’t have the advantage of being an
> object in memory ready for sending to the immediate mode graphics API
> for painting, but it doesn’t sound like your use case needs this.
>
> > Developers have told me that they want mouse events delegated
> > to the shadow dom, based on paths they've bound.
>
> Can you explain this a bit more?  I didn’t quite follow.  Does it mean
> you want to have it so that
>
>   * you associate an element in the <canvas> subtree with a particular
>     region of the canvas (by giving it a path, say)
>   * mousedown/mouseup/etc. events get dispatched to the <canvas> subtree
>     element instead of the <canvas> itself
>
> ?
>
Not wanting to respond for Charles necessarily, but yes. The current,
appropriate implementation for keyboard events is that they route to the
canvas subtree. This would allow the author to fully bind the drawing
object to the same DOM element that would process the keyboard for it. It
is a lighter weight solution than full retained mode graphics that I was
proposing earlier.

Once a path is registered with canvas, the user agent receives mouse events
and assess where they "hit" within the registered draw paths (that also
include Z order) and dispatches the events to the <canvas> subtree element.

> > Part 2. Creating a CanvasPath object and stringify methods in the spec.
> >
> > I've done a basic supplemental. setElementFromPath binds the current
> > path onto an element, with an optional zIndex. I've also done a
stringify,
> > supplemental, which would make life a lot easier on developers when it
> > comes to getting a string for the current path or transformation
matrix.
> >
> > // CanvasPath extension
> > [Supplemental]
> > interface CanvasRenderingContext2D {
> >     void beginFromPath(in CanvasPath path) ;
> >     void setElementPath(in Element element, in optional DOMString
zIndex) ;
> >     CanvasPath getCurrentPath();
> > }
>
> So, setElementPath() associates the canvas context’s current path with
> that element.  Apart from the automatic event dispatching thing you
> mention above, what effect does this association have?  (Sorry if this
> is obvious, since I don’t have the context here.)
>
This would also allow us a user agent to provide the physical bounds of the
drawing object to the object in the fallback content that supplies
information to platform accessibility APIs. This is essential for screen
magnification and for driving Braille devices by a screen reader. Chuck is
part of the canvas HTML accessibility sub team and we are attempting to
bind an accessibility requirement to a mainstream need for hit testing.

> > // Stringify extensions, defined by SVG normalized paths and CSSMatrix.
> > [Supplemental]
> > interface CanvasRenderingContext2D {
> >     void beginFromPath(in normalizedPathSegList path) ;
>
> There’s no type normalizedPathSegList – that’s the name of a property on
> SVGPathElement objects.  So I guess you mean SVGPathSegList, but as I
> mention above there’s no way to create independent SVGPathSegList
> objects in SVG (at the moment).  You can only grab them off existing
> <path> elements.
>
> >     void beginFromPath(in DOMString path) ;
>
> Being able to set the canvas context’s current path using SVG path data
> syntax seems like a nice idea.
>
> > Stringify options allow developers more room for loose coupling of
events;
>
> I don’t really understand what loose coupling of events means, here.
>
> > they also allow better interop between SVG and Canvas... They help
> > with debugging;
> > they can reduce the overhead of complex objects, by reducing the
> > number of method
> > calls, considerably.
>
> How about just using the SVG path data string itself? :-)
>
> --
> Cameron McCormack ≝ http://mcc.id.au/