Re: hit testing and retained graphics

So, we seem to have submitted a lot of requirements on the need for
retained mode graphic drawing paths  that would allow for:

- hit testing where the pointer events are sent to the canvas subtree
element bound to it
- the supplying of accessible object bounds of visual drawing objects.

I spoke with Charles Pritchard today and reason we need it.

A new rather severe use case we have not addressed is mobile devices and
accessibility. VoiceOver responds to a user's being able to announce the
object under which a user point their finger while moving over the screen.
Then when a user is able to tap to activate the underlying object. Without
know the bound of the object represented in fallback content VoiceOver is
unable to determine whether the finger is within the drawing object. This
will create a similar problem for Android. Currently, VoiceOver announces
image.

I have cc'd James Craig from Apple's VoiceOver team to weigh in on this.

Also, Tab you have come to the W3C public list to gather use cases, etc.
What are doing with the information. I need to know as otherwise I will
need to write my own change proposal for the W3C 2d Canvas API to fill
address this issue. We are running out of time in terms of getting issues
registered and change proposals submitted. I would prefer to work together
vs. writing competing change proposals that would do achieve similar
results.

Without this canvas is an accessibility hole on both desktop and mobile for
screen reading and screen magnification.

Rich

Rich Schwerdtfeger
CTO Accessibility Software Group

public-canvas-api-request@w3.org wrote on 07/06/2011 08:22:31 AM:

> From: Richard Schwerdtfeger/Austin/IBM@IBMUS
> To: Steve Faulkner <faulkner.steve@gmail.com>
> Cc: Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com>, Cameron
> McCormack <cam@mcc.id.au>, Charles McCathieNevile
> <chaals@opera.com>, Charles Pritchard <chuck@jumis.com>, Cynthia
> Shelly <cyns@microsoft.com>, "david.bolter@gmail.com"
> <david.bolter@gmail.com>, "E.J. Zufelt" <everett@zufelt.ca>, Frank
> Olivier <Frank.Olivier@microsoft.com>, Henri Sivonen
> <hsivonen@iki.fi>, "Tab Atkins Jr." <jackalmage@gmail.com>, John
> Foliot <jfoliot@stanford.edu>, "Mike@w3.org" <Mike@w3.org>, Paul
> Bakaus <pbakaus@zynga.com>, "public-canvas-api@w3.org" <public-
> canvas-api@w3.org>, "public-html@w3.org" <public-html@w3.org>,
> "public-html-a11y@w3.org" <public-html-a11y@w3.org>, Sean Hayes
> <Sean.Hayes@microsoft.com>
> Date: 07/06/2011 08:26 AM
> Subject: Re: hit testing and retained graphics
> Sent by: public-canvas-api-request@w3.org
>
> Yeah, that is unlikely to happen (re-rendering with font smoothing)
> with canvas.
>
>
> Rich Schwerdtfeger
> CTO Accessibility Software Group
>
> [image removed] Steve Faulkner ---07/03/2011 07:38:21 AM---Hi ben,
> Also, in practice, such AT tends to have additional features like
>
> From: Steve Faulkner <faulkner.steve@gmail.com>
> To: Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com>
> Cc: Charles Pritchard <chuck@jumis.com>, Henri Sivonen
> <hsivonen@iki.fi>, Sean Hayes <Sean.Hayes@microsoft.com>, "E.J.
> Zufelt" <everett@zufelt.ca>, Paul Bakaus <pbakaus@zynga.com>, "Tab
> Atkins Jr." <jackalmage@gmail.com>, John Foliot
> <jfoliot@stanford.edu>, Charles McCathieNevile <chaals@opera.com>,
> Richard Schwerdtfeger/Austin/IBM@IBMUS, Cameron McCormack
> <cam@mcc.id.au>, Cynthia Shelly <cyns@microsoft.com>,
> "david.bolter@gmail.com" <david.bolter@gmail.com>, Frank Olivier
> <Frank.Olivier@microsoft.com>, "Mike@w3.org" <Mike@w3.org>, "public-
> canvas-api@w3.org" <public-canvas-api@w3.org>, "public-html@w3.org"
> <public-html@w3.org>, "public-html-a11y@w3.org" <public-html-a11y@w3.org>
> Date: 07/03/2011 07:38 AM
> Subject: Re: hit testing and retained graphics
>
>
>
>
> Hi ben,
> Also, in practice, such AT tends to have additional features like
> reading out the text in focus, re-rendering the text at higher zoom.
> Remote AT is better placed to do those things,
> and I think users would expect those features to work. So I'm not sure
> of the advantage of taking a slice of current functionality and trying
> to deliver that to local AT?
>
> it is true screen magnifiers come in all shapes and sizes, i myself
> have zoomtext 9.1 magnifier/reader. I have friends who have only the
> magnifier version as they don't want or require the reading function.
>
> Providing such a basic feature as programatic focus on areas of the
> canvas will be the difference between interactive canvas content
> being usable or unusable for some users.
>
> regards
> stevef
>
>
> On 3 July 2011 10:49, Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com
> > wrote:
> On Sun, Jul 3, 2011 at 7:29 AM, Steve Faulkner <faulkner.steve@gmail.com
> > wrote:
> > I thought I was clear, I have been talking about the screen magnifier
case ,
> > not the general AT case. The only information that would need to be
provided
> > from the remote machine is the size and position of the focused object,
this
> > could then be used to provide the focus information to the local
> > accessibility layer.
>
> You're still going to need custom code to get this information out of
> custom views
> on the remote side.
>
> Also, in practice, such AT tends to have additional features like
> reading out the text in focus, re-rendering the text at higher zoom.
> Remote AT is better placed to do those things,
> and I think users would expect those features to work. So I'm not sure
> of the advantage of taking a slice of current functionality and trying
> to deliver that to local AT?
>
> --
> Benjamin Hawkes-Lewis
>
>
>
> --
> with regards
>
> Steve Faulkner
> Technical Director - TPG
>
> www.paciellogroup.com | www.HTML5accessibility.com |
> www.twitter.com/stevefaulkner
> HTML5: Techniques for providing useful text alternatives -
> dev.w3.org/html5/alt-techniques/
> Web Accessibility Toolbar -
www.paciellogroup.com/resources/wat-ie-about.html