Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility

Oliver,

What's your take on the VoiceOver integration with canvas?
First and foremost, we're looking to enable authors to provide more information to accessibility software.

This is a brainstorming thread.

Ideas welcome.

On Mar 30, 2011, at 12:20 PM, Oliver Hunt <oliver@apple.com> wrote:

> 
> On Mar 30, 2011, at 12:06 PM, Richard Schwerdtfeger wrote:
> 
>> Oliver,
>> 
>> It is an attempt to simplify hit testing in canvas while at the same time providing a vehicle to tell an assistive technology the bounds of the corresponding UI object being drawn on canvas as represented in fallback content. Currently there is no mapping. These bounds are needed by screen magnifiers for zooming and screen readers for Braille support (earlier post). 
>> 
>> Now the canvas author has to manage all the hit testing. This is a canvas deficiency that should have been in place to start - so grafting seems like an inappropriate response. Consequently, I am also seeing canvas applications that create the equivalent of visio by creating separate canvas elements overlayed on top of another canvas to represent drawing objects. This is very inefficient and will get worse unless something is done.
>> 
> This is not a canvas deficiency -- canvas is an immediate mode renderer, one of the things you have to handle yourself when dealing with _any_ immediate mode renderer is hit detection.  This is true of Canvas, GDI, CG, raw component painting in Java, etc, etc
> 
> If you want hit detection to be done for you, you want a retained mode renderer, such as SVG.
> 
> --Oliver
>> 
>> 
>> Rich Schwerdtfeger
>> CTO Accessibility Software Group
>> 
>> <graycol.gif>Oliver Hunt ---03/30/2011 12:11:53 PM---This feels like an attempt to graft retained mode rendering on to canvas, what am i missing? --Olive
>> 
>> From:	Oliver Hunt <oliver@apple.com>
>> To:	Richard Schwerdtfeger/Austin/IBM@IBMUS
>> Cc:	Boris Zbarsky <bzbarsky@mit.edu>, chuck@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com, faulkner.steve@gmail.com, franko@microsoft.com, public-canvas-api@w3.org, public-html@w3.org, public-html-a11y@w3.org, public-html-request@w3.org
>> Date:	03/30/2011 12:11 PM
>> Subject:	Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility
>> Sent by:	public-canvas-api-request@w3.org
>> 
>> 
>> 
>> This feels like an attempt to graft retained mode rendering on to canvas, what am i missing?
>> 
>> --Oliver
>> 
>> On Mar 30, 2011, at 8:39 AM, Richard Schwerdtfeger wrote:
>> 
>> Thanks Boris. Sorry for being a pest but I really want developers to work through the issues. 
>> 
>> So, you would do the following:
>> 
>> - Assign the a closed draw path to an element in fallback content: setClickableRegion(element). This immediately makes the association and places it at the bottom of the list.
>> - Any time you draw the element it moves it to the top of the list.
>> - If the fallback element is removed the association would need to go away. I did not address that so I will need to add that to the proposal
>> - When a pointer event (click, etc.) goes to the fallback element the normal capture/bubbling event processing would apply
>> 
>> Rich
>> 
>> 
>> Rich Schwerdtfeger
>> CTO Accessibility Software Group
>> 
>> <graycol.gif>Boris Zbarsky ---03/30/2011 10:05:48 AM---On 3/30/11 10:55 AM, Richard Schwerdtfeger wrote: > Seeing as nobody has commented can we assume tha
>> 
>> From: Boris Zbarsky <bzbarsky@mit.edu>
>> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
>> Cc: public-canvas-api@w3.org, public-html@w3.org, public-html-a11y@w3.org, public-html-request@w3.org, chuck@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com, faulkner.steve@gmail.com, franko@microsoft.com
>> Date: 03/30/2011 10:05 AM
>> Subject: Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility
>> 
>> 
>> 
>> 
>> On 3/30/11 10:55 AM, Richard Schwerdtfeger wrote:
>> > Seeing as nobody has commented can we assume that developers have no
>> > problem with our totally changing the canvas 2D API to support clickable
>> > regions?
>> 
>> It might just mean that people don't read their mailing list spam every 
>> few hours.
>> 
>> >From reading over your proposal, I'm not sure I follow how it would 
>> behave in the face of DOM mutations (e.g. elements being removed from 
>> the fallback content).
>> 
>> Or put another way, once you draw, you're permanently attaching some 
>> element to the canvas, right? Or is that not the proposal?
>> 
>> -Boris
>> 
>