Re: feedback requested: Canvas change for improved hit testing that also facilitates accessibility

Oliver,

GDI provides and infrastructure that supports hit testing in Windows. The
bounding rectangle of each device context is bound to the object and used
in hit testing along with Z-order, etc. What I am telling you is that many
developers are using canvas beyond the limitations and not going to SVG
because for people working in the Windows space Canvas is closer to what
they know. I am also saying that we need a vehicle for supplying the bounds
of a UI object for accessibility. This has, on Windows, been tied to this
device context bounding rectangle.

>From an accessibility perspective telling a developer they should not do
something in canvas easily so go use SVG does not work.

Rich

Rich Schwerdtfeger
CTO Accessibility Software Group



From:	Oliver Hunt <oliver@apple.com>
To:	Richard Schwerdtfeger/Austin/IBM@IBMUS
Cc:	Boris Zbarsky <bzbarsky@mit.edu>, chuck@jumis.com,
            cyns@exchange.microsoft.com, david.bolter@gmail.com,
            faulkner.steve@gmail.com, franko@microsoft.com,
            public-canvas-api@w3.org, public-canvas-api-request@w3.org,
            public-html@w3.org, public-html-a11y@w3.org,
            public-html-request@w3.org
Date:	03/30/2011 02:20 PM
Subject:	Re: feedback requested: Canvas change for improved hit testing
            that also     facilitates accessibility




On Mar 30, 2011, at 12:06 PM, Richard Schwerdtfeger wrote:



      Oliver,

      It is an attempt to simplify hit testing in canvas while at the same
      time providing a vehicle to tell an assistive technology the bounds
      of the corresponding UI object being drawn on canvas as represented
      in fallback content. Currently there is no mapping. These bounds are
      needed by screen magnifiers for zooming and screen readers for
      Braille support (earlier post).

      Now the canvas author has to manage all the hit testing. This is a
      canvas deficiency that should have been in place to start - so
      grafting seems like an inappropriate response. Consequently, I am
      also seeing canvas applications that create the equivalent of visio
      by creating separate canvas elements overlayed on top of another
      canvas to represent drawing objects. This is very inefficient and
      will get worse unless something is done.


This is not a canvas deficiency -- canvas is an immediate mode renderer,
one of the things you have to handle yourself when dealing with _any_
immediate mode renderer is hit detection.  This is true of Canvas, GDI, CG,
raw component painting in Java, etc, etc

If you want hit detection to be done for you, you want a retained mode
renderer, such as SVG.

--Oliver




      Rich Schwerdtfeger
      CTO Accessibility Software Group

      <graycol.gif>Oliver Hunt ---03/30/2011 12:11:53 PM---This feels like
      an attempt to graft retained mode rendering on to canvas, what am i
      missing? --Olive

      From: Oliver Hunt <oliver@apple.com>
      To: Richard Schwerdtfeger/Austin/IBM@IBMUS
      Cc: Boris Zbarsky <bzbarsky@mit.edu>, chuck@jumis.com,
      cyns@exchange.microsoft.com, david.bolter@gmail.com,
      faulkner.steve@gmail.com, franko@microsoft.com,
      public-canvas-api@w3.org, public-html@w3.org, public-html-a11y@w3.org
      , public-html-request@w3.org
      Date: 03/30/2011 12:11 PM
      Subject: Re: feedback requested: Canvas change for improved hit
      testing that also facilitates accessibility
      Sent by: public-canvas-api-request@w3.org





      This feels like an attempt to graft retained mode rendering on to
      canvas, what am i missing?

      --Oliver

      On Mar 30, 2011, at 8:39 AM, Richard Schwerdtfeger wrote:

            Thanks Boris. Sorry for being a pest but I really want
            developers to work through the issues.

            So, you would do the following:

            - Assign the a closed draw path to an element in fallback
            content: setClickableRegion(element). This immediately makes
            the association and places it at the bottom of the list.
            - Any time you draw the element it moves it to the top of the
            list.
            - If the fallback element is removed the association would need
            to go away. I did not address that so I will need to add that
            to the proposal
            - When a pointer event (click, etc.) goes to the fallback
            element the normal capture/bubbling event processing would
            apply

            Rich


            Rich Schwerdtfeger
            CTO Accessibility Software Group

            <graycol.gif>Boris Zbarsky ---03/30/2011 10:05:48 AM---On
            3/30/11 10:55 AM, Richard Schwerdtfeger wrote: > Seeing as
            nobody has commented can we assume tha

            From: Boris Zbarsky <bzbarsky@mit.edu>
            To: Richard Schwerdtfeger/Austin/IBM@IBMUS
            Cc: public-canvas-api@w3.org, public-html@w3.org,
            public-html-a11y@w3.org, public-html-request@w3.org,
            chuck@jumis.com, cyns@exchange.microsoft.com,
            david.bolter@gmail.com, faulkner.steve@gmail.com,
            franko@microsoft.com
            Date: 03/30/2011 10:05 AM
            Subject: Re: feedback requested: Canvas change for improved hit
            testing that also facilitates accessibility




            On 3/30/11 10:55 AM, Richard Schwerdtfeger wrote:
            > Seeing as nobody has commented can we assume that developers
            have no
            > problem with our totally changing the canvas 2D API to
            support clickable
            > regions?

            It might just mean that people don't read their mailing list
            spam every
            few hours.

            >From reading over your proposal, I'm not sure I follow how it
            would
            behave in the face of DOM mutations (e.g. elements being
            removed from
            the fallback content).

            Or put another way, once you draw, you're permanently attaching
            some
            element to the canvas, right? Or is that not the proposal?

            -Boris