Re: hit testing and retained graphics

Rich Schwerdtfeger
CTO Accessibility Software Group

public-html-request@w3.org wrote on 06/28/2011 09:35:20 AM:

> From: Silvia Pfeiffer <silviapfeiffer1@gmail.com>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc: Charles McCathieNevile <chaals@opera.com>, Cameron McCormack
> <cam@mcc.id.au>, Charles Pritchard <chuck@jumis.com>, Cynthia Shelly
> <cyns@microsoft.com>, david.bolter@gmail.com, Frank Olivier
> <Frank.Olivier@microsoft.com>, "Tab Atkins Jr."
> <jackalmage@gmail.com>, Mike@w3.org, public-canvas-api@w3.org,
> public-html@w3.org, public-html-a11y@w3.org,
public-html-a11y-request@w3.org
> Date: 06/28/2011 09:37 AM
> Subject: Re: hit testing and retained graphics
> Sent by: public-html-request@w3.org
>
> On Mon, Jun 27, 2011 at 10:41 PM, Richard Schwerdtfeger
> <schwer@us.ibm.com> wrote:
> >
> >
> > Rich Schwerdtfeger
> > CTO Accessibility Software Group
> >
> > public-html-a11y-request@w3.org wrote on 06/27/2011 06:20:44 AM:
> >
> >> From: "Charles McCathieNevile" <chaals@opera.com>
> >> To: Richard Schwerdtfeger/Austin/IBM@IBMUS, "Tab Atkins Jr."
> >> <jackalmage@gmail.com>
> >> Cc: "Cameron McCormack" <cam@mcc.id.au>, "Charles Pritchard"
> >> <chuck@jumis.com>, "Cynthia Shelly" <cyns@microsoft.com>,
> >> david.bolter@gmail.com, "Frank Olivier"
> >> <Frank.Olivier@microsoft.com>, Mike@w3.org, public-canvas-
> >> api@w3.org, public-html@w3.org, public-html-a11y@w3.org
> >> Date: 06/27/2011 06:22 AM
> >> Subject: Re: hit testing and retained graphics
> >> Sent by: public-html-a11y-request@w3.org
> >>
> >> On Thu, 23 Jun 2011 22:28:32 +0200, Tab Atkins Jr.
<jackalmage@gmail.com>
> >> wrote:
> >>
> >> > On Thu, Jun 23, 2011 at 12:56 PM, Richard Schwerdtfeger
> >> > <schwer@us.ibm.com> wrote:
> >> >>> So normally, I imagine, hit testing would be done either by using
> >> >>> isPointInPath() or by custom code looking at a mouse event’s x/y
> >> >>> values.
> >> >>> I think this proposal doesn’t work with isPointInPath(), though,
is
> >> >>> that
> >> >>> right?
> >> >>>
> >> >> I think it would but we would need to incorporate Z order and a
notion
> >> >> of the last drawn element to compute which element is on top. The
user
> >> >> agent would need to manage this.
> >> >
> >> > You are attempting to recreate a retained-mode API in an
> >> > immediate-mode API. Why is "use SVG" not sufficient for this?
> >>
> >> Because people don't - they use canvas instead. If that were not the
case,
> >> the whole effort to specify canvas would be solving a theoretical
problem.
> >>
> >
> > So, one of the main reasons that developers use canvas is because it is
> > close to the type of API you see in Windows GDI and has been around
since
> > before 1990. SVG, while having many benefits will take a long time to
be
> > adopted by the development community in mass. Meanwhile, canvas will be
used
> > by developers. When a cross-platform 2D API was created for the Web it
was
> > like a boom for developers. This is what they are used to. Once the cat
is
> > let out of the bag you can't stop developers from using it. It is
supported
> > in all the browsers. However, now the fact that it is attached to HTML
and
> > the runway is shortening to make it accessible we have the JavaScript
> > accessibility problem. It will create a huge black eye on HTML as it is
> > inaccessible. It's use will now need to be prohibited for governments
much
> > the same way that JavaScript was prohibited for years. The difference
being
> > we know how to fix it and could do something about it before the
problem
> > gets worse.
>
> What about Adobe Flash in this area? Adobe Flash is used for many of
> the same use cases that canvas is used for. Is Flash more accessible?
> How do they do it? Is Flash prohibited because it's not accessible?
>
Hi Silvia,

Actually, I know quite a bit about this subject. Flash, internally
implements MSAA on their drawing objects. It is very primitive. Currently,
Flash does not have a declarative infrastructure under Flash to map drawing
objects to an accessibility API so the author must do this by hand. They
also have a limitation on the API that they support today. Adobe publicly
stated that they will be addressing this problem in a future release and
will be, in fact, implementing IAccessible2. But, all this does is
replicate what is already done in browsers today. Your browser for Chrome,
FF, and Safari on Windows already bind DOM objects to MSAA/IAccessible2
APIs. The advantage we have in canvas that would actually benefit from an
implementation on top of canvas where it could bind DOM objects to drawing
objects in canvas.

I would argue that the approach we are taking on canvas is probably the
most innovative approaches to date, to making graphical drawings on canvas
accessible, in that we can use declarative markup in the shadow DOM to
represent drawing objects on canvas and we can add ARIA to it to provide
the declarative semantics. You can not only pick up element semantics but
also structural semantics from the DOM. These are huge advantages over
Flash accessibility where the author has to implement the accessibility API
in flash content. If you use standard FLEX controls the FLEX authors have
done alot of this for you and what they need to make it better is add
document structure and a richer API set - something we already have in
browsers.

I would argue that Flash could actually benefit by building on top of
canvas with our accessibility infrastructure.

> On a side not: I'm wondering if in the majority of cases we may be
> trying to achieve the impossible. For example, you may try as hard as
> you want, but you will not achieve it that a blind user will be able
> to drive a car with nothing but machine support. Even with the best
> technology that will communicate what is happening around them, it
> will be impossible to provide a description of the visible environment
> sufficiently timely to make it possible/safe to drive without seeing.
> What is our solution for the impossible situation?
>
My response is you are attempting to argue a point by stating we need to
boil an ocean to solve the cases that are needed to keep people of
disabilities employed. That is unnecessary. To address what you want is an
entirely new research effort. At times you need to provide alternative
equivalents for specific users. This is not a new concept.

If we want a blind person to be able to use a car to get around, design a
car that drives itself ... but I think Google has done that. :-)

> Regards,
> Silvia.
>