Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility

The first result i find for hit testing in GDI is http://msdn.microsoft.com/en-us/library/ms969920.aspx

Which implies to me that they expect a developer to do their own hit testing.

I am surprised that GDI provides any automatic hit testing for drawing to an HDC, but before commenting further I need to see an example of the API, can you point me at documentation or code that uses it?

--Oliver

On Mar 30, 2011, at 1:51 PM, Richard Schwerdtfeger wrote:

> Oliver,
> 
> GDI provides and infrastructure that supports hit testing in Windows. The bounding rectangle of each device context is bound to the object and used in hit testing along with Z-order, etc. What I am telling you is that many developers are using canvas beyond the limitations and not going to SVG because for people working in the Windows space Canvas is closer to what they know. I am also saying that we need a vehicle for supplying the bounds of a UI object for accessibility. This has, on Windows, been tied to this device context bounding rectangle. 
> 
> From an accessibility perspective telling a developer they should not do something in canvas easily so go use SVG does not work. 
> 
> Rich
> 
> 
> Rich Schwerdtfeger
> CTO Accessibility Software Group
> 
> <graycol.gif>Oliver Hunt ---03/30/2011 02:20:27 PM---On Mar 30, 2011, at 12:06 PM, Richard Schwerdtfeger wrote: > Oliver,
> 
> From:	Oliver Hunt <oliver@apple.com>
> To:	Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc:	Boris Zbarsky <bzbarsky@mit.edu>, chuck@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com, faulkner.steve@gmail.com, franko@microsoft.com, public-canvas-api@w3.org, public-canvas-api-request@w3.org, public-html@w3.org, public-html-a11y@w3.org, public-html-request@w3.org
> Date:	03/30/2011 02:20 PM
> Subject:	Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility
> 
> 
> 
> 
> On Mar 30, 2011, at 12:06 PM, Richard Schwerdtfeger wrote:
> 
> Oliver,
> 
> It is an attempt to simplify hit testing in canvas while at the same time providing a vehicle to tell an assistive technology the bounds of the corresponding UI object being drawn on canvas as represented in fallback content. Currently there is no mapping. These bounds are needed by screen magnifiers for zooming and screen readers for Braille support (earlier post). 
> 
> Now the canvas author has to manage all the hit testing. This is a canvas deficiency that should have been in place to start - so grafting seems like an inappropriate response. Consequently, I am also seeing canvas applications that create the equivalent of visio by creating separate canvas elements overlayed on top of another canvas to represent drawing objects. This is very inefficient and will get worse unless something is done.
> 
> This is not a canvas deficiency -- canvas is an immediate mode renderer, one of the things you have to handle yourself when dealing with _any_ immediate mode renderer is hit detection. This is true of Canvas, GDI, CG, raw component painting in Java, etc, etc
> 
> If you want hit detection to be done for you, you want a retained mode renderer, such as SVG.
> 
> --Oliver
> 
> 
> 
> Rich Schwerdtfeger
> CTO Accessibility Software Group
> 
> <graycol.gif>Oliver Hunt ---03/30/2011 12:11:53 PM---This feels like an attempt to graft retained mode rendering on to canvas, what am i missing? --Olive
> 
> From: Oliver Hunt <oliver@apple.com>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc: Boris Zbarsky <bzbarsky@mit.edu>, chuck@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com, faulkner.steve@gmail.com, franko@microsoft.com, public-canvas-api@w3.org, public-html@w3.org, public-html-a11y@w3.org, public-html-request@w3.org
> Date: 03/30/2011 12:11 PM
> Subject: Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility
> Sent by: public-canvas-api-request@w3.org
> 
> 
> 
> 
> This feels like an attempt to graft retained mode rendering on to canvas, what am i missing?
> 
> --Oliver
> 
> On Mar 30, 2011, at 8:39 AM, Richard Schwerdtfeger wrote:
> 
> Thanks Boris. Sorry for being a pest but I really want developers to work through the issues. 
> 
> So, you would do the following:
> 
> - Assign the a closed draw path to an element in fallback content: setClickableRegion(element). This immediately makes the association and places it at the bottom of the list.
> - Any time you draw the element it moves it to the top of the list.
> - If the fallback element is removed the association would need to go away. I did not address that so I will need to add that to the proposal
> - When a pointer event (click, etc.) goes to the fallback element the normal capture/bubbling event processing would apply
> 
> Rich
> 
> 
> Rich Schwerdtfeger
> CTO Accessibility Software Group
> 
> <graycol.gif>Boris Zbarsky ---03/30/2011 10:05:48 AM---On 3/30/11 10:55 AM, Richard Schwerdtfeger wrote: > Seeing as nobody has commented can we assume tha
> 
> From: Boris Zbarsky <bzbarsky@mit.edu>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc: public-canvas-api@w3.org, public-html@w3.org, public-html-a11y@w3.org, public-html-request@w3.org, chuck@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com, faulkner.steve@gmail.com, franko@microsoft.com
> Date: 03/30/2011 10:05 AM
> Subject: Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility
> 
> 
> 
> 
> On 3/30/11 10:55 AM, Richard Schwerdtfeger wrote:
> > Seeing as nobody has commented can we assume that developers have no
> > problem with our totally changing the canvas 2D API to support clickable
> > regions?
> 
> It might just mean that people don't read their mailing list spam every 
> few hours.
> 
> >From reading over your proposal, I'm not sure I follow how it would 
> behave in the face of DOM mutations (e.g. elements being removed from 
> the fallback content).
> 
> Or put another way, once you draw, you're permanently attaching some 
> element to the canvas, right? Or is that not the proposal?
> 
> -Boris
>