RE: hit testing and retained graphics

I have some concerns about this. It means:

- The author has to do the transformations
- We have to add DOM attributes
- The mouse events are routed to canvas and not the sub-DOM elements where
the keyboard handling is going on. The user agent could process the mouse
events at canvas and then propagate them to the corresponding DOM object in
the subtree.

Your proposal does have the advantage that the bounds of the objects are in
the DOM but for HTML we don't have this for any of the DOM elements now.

What I was thinking was the following:

- Today Canvas has the notion of a context
- We allow the author to have the same context (with methods) for each
drawing object and apply a bounds and z-index as you suggest.
- We then bind each drawing object to the canvas subtree DOM element:

So each drawing object would be an instance of a canvas context with
methods were we do something like:

1. we assume that the canvas element when the page is created is an
instance of a canvasObject (having a context)
2. we assume that drawingOjects are a subclass of canvasObject that support
all the canvas2DAPI in canvasObject with some additions such as:
    - ZIndex attribute
    - a bounding drawing path and methods for modifying them
    - a method for associating the drawingObject with a canvas subtree DOM
element.
3. we add an method to canvas that says addDrawObject.
On the canvas element we have the following:

var canvas = document.getElementById(‘canvasTest’);
if (canvas.getContext){
var ctx = canvas.getContext(’2d’);
DO = new drawingObject();
dctx= DO.getContext('2d');
dctx.ZIndex="2";
dctx.beginPath();
dctx.moveTo(130,100);
dctx.lineTo...
...
dctx.setPathtoBounds();
dctx.setDOMSubtreeNode(foo); //Foo is a subtree node where keyboard events
go to and we do our accessibility  enablement to populate the accessibility
tree
                                                    //Internally the user
agents maps the bounding rectangle for the accessible of the DOM object as
a best fit rectangle of the
                                                    //path used to form the
bounds of the drawing object
ctx.addDrawingObject(DO);
}

Now the user agent the information needed to do the hit testing and
retained mode graphics (I am oversimplifying for illustration purposes) to
be able to track the pointing device input and routing it to the same DOM
objects that process the keyboard and all the other accessibility
information. This includes hit testing.

Mike provided feedback on the HTML A11Y call that authors did not want to
do the hit testing themselves.
Rich Schwerdtfeger
CTO Accessibility Software Group



From: Frank Olivier <Frank.Olivier@microsoft.com>
To: Richard Schwerdtfeger/Austin/IBM@IBMUS, "chuck@jumis.com"
            <chuck@jumis.com>, "Mike@w3.org" <Mike@w3.org>,
            "david.bolter@gmail.com" <david.bolter@gmail.com>, Cynthia
            Shelly <cyns@microsoft.com>
Cc: "public-canvas-api@w3.org" <public-canvas-api@w3.org>,
            "public-html-a11y@w3.org" <public-html-a11y@w3.org>,
            "public-html@w3.org" <public-html@w3.org>
Date: 06/20/2011 11:01 AM
Subject: RE: hit testing and retained graphics
Sent by: public-html-request@w3.org



I would leave hit testing up to the (javascript) author. I would recommend
that they set existing x,y position, z-index attributes on the DOM objects
in the canvas subtree to report what the UI 'looks like' to AT tools. This
way, the AT tools don't need to change - this part of the DOM is no
different to them than any other part - and authors need to be annotating
canvas DOM objects with correct information anyway (labels, aria
attributes, etc).

From: Richard Schwerdtfeger [mailto:schwer@us.ibm.com]
Sent: Friday, June 17, 2011 11:42 AM
To: chuck@jumis.com; Frank Olivier; Mike@w3.org; david.bolter@gmail.com;
Cynthia Shelly
Cc: public-canvas-api@w3.org; public-html-a11y@w3.org; public-html@w3.org
Subject: hit testing and retained graphics

Charles, Frank, Mike,

I am back from vacation. How far do we need to go with hit testing? Right
now I am looking at associating a closed draw path with a DOM object in the
canvas subtree. We would then need to address the routing of pointing
device input events to the DOM object. The drawing path can be used to
provide bound information to platform accessibility API.

Do we need to bind any other drawing properties to the canvas object -
similar to the way device context's are handled on graphic subsystems like
Windows?

Mike, I am including you as before I went on vacation you indicated that a
number of developers desired this feature and wanted to be involved.

Rich


Rich Schwerdtfeger
CTO Accessibility Software Group