RE: hit testing and retained graphics

Frank,

We met in Redmond and we agreed that we needed the ability to do hit
testing and how to do it so that we could bind the drawing object paths to
the accessibility API. We do NOT need to implement full retained mode
graphics to do this.

If we cannot make canvas accessible, which we can't without the ability to
supply bounding object information then canvas MUST be removed from HTML 5.
We cannot continue to put half-baked accessible solutions into the standard
or browsers.

The browser manufactures put canvas into the spec. and added support for it
but right now it is like an inaccessible bolt on to a specification that
the entire world is going to be using. SVG has a VERY long way to go for
accessibility and in fact, the tremendous amount of gas wasted trying to
make canvas and the rest of HTML accessible on things like longdesc, and
alt text, wording on how ARIA is integrated have delayed work on
accessibility resources by over 2 years wrt. SVG. Starting on SVG now is
like having a kid who does not want to finish mowing the lawn because they
have something more interesting to work on.

The reality is canvas is we have a half pregnant solution with canvas and
we need to finish it. Right now developers are producing inaccessible
canvas solutions because the browser manufactures slammed an incomplete
solution into the HTML standard that everyone on the planet uses today. We
are addressing accessibility late in the design cycle and, frankly, I
personally have been herding cats on browser manufactures to clean up the
mess they created.

If browser manufacturers do not want canvas, and that includes Microsoft,
then say so now so that we don't deliver an inaccessible solution to the
world and gut it from the spec. I will go to the W3C CEO and the chairs
within the next week and demand that the thing be removed.

We are setting HTML up to be another JavaScript accessibility problem. If
HTML is not accessible it is not going to be accepted.

As for SVG, we need to finish HTML accessibility and get ARIA through CR
before we can even think of tackling the accessibility of something as big
as SVG. That is why SVG accessibility was moved to ARIA 2.0. To properly
fix SVG accessibility and standardize it we are looking at a 3 year
standards effort.

Let's get canvas done first with the minimal amount of work. The only way
to do that is to add hit testing  on bounding region draw path assignments,
skip full retained mode graphics, and fix some of the stragglers around
Maciej's request on focus ring and adding text baseline support. Frankly,
the time spent waffling on getting this stuff done than it would take to
fix the thing.

So, Frank it appears canvas accessibility is in the Microsoft's hands now.
Do we cut it from HTML or do we fix it? If, we leave it in browsers
unfixed, I will point to the browser manufacturers for leaving it that way
when asked by government officials why it is not accessible.

Rich

Rich Schwerdtfeger
CTO Accessibility Software Group



From: Frank Olivier <Frank.Olivier@microsoft.com>
To: Richard Schwerdtfeger/Austin/IBM@IBMUS, "Tab Atkins Jr."
            <jackalmage@gmail.com>
Cc: Cameron McCormack <cam@mcc.id.au>, Charles Pritchard
            <chuck@jumis.com>, Cynthia Shelly <cyns@microsoft.com>,
            "david.bolter@gmail.com" <david.bolter@gmail.com>,
            "Mike@w3.org" <Mike@w3.org>, "public-canvas-api@w3.org"
            <public-canvas-api@w3.org>, "public-html@w3.org"
            <public-html@w3.org>, "public-html-a11y@w3.org"
            <public-html-a11y@w3.org>
Date: 06/23/2011 06:31 PM
Subject: RE: hit testing and retained graphics
Sent by: public-html-request@w3.org



Microsoft considers SVG to be the correct way to do retained mode graphics.
We have no plans to add retained capabilities to the immediate mode canvas
API. If SVG doesn’t address the problem then effort would be better
expended on fixing that problem.

Thanks
Frank


From: Richard Schwerdtfeger [mailto:schwer@us.ibm.com]
Sent: Thursday, June 23, 2011 2:29 PM
To: Tab Atkins Jr.
Cc: Cameron McCormack; Charles Pritchard; Cynthia Shelly;
david.bolter@gmail.com; Frank Olivier; Mike@w3.org;
public-canvas-api@w3.org; public-html@w3.org; public-html-a11y@w3.org
Subject: Re: hit testing and retained graphics

Hi Tab,

We are trying to make canvas accessible and we are trying to provide hit
testing support in canvas vs. throwing out canvas for SVG. SVG has a VERY
long way before it is accessible and canvas developers, who have canvas
implementations, don't want to throw all their work away to get hit testing
by using SVG.

I have a very large IBM application in development and SVG given the state
of SVG accessibility and how the application is constructed it is not a
good fit. I also have a commitment to ensure HTML 5 is fully accessible, as
part of the HTML accessibility task force effort, and we need to make
magnifiers aware of drawing object location and dimensions in canvas.

Also, there is a need to have hit testing support in canvas and I am trying
to marry an accessibility feature with a mainstream feature developers
need.

I had suggested full retained mode graphics by creating drawing objects
that would inherit the 2d Context API of canvas, however the discussion on
the list was indicated that was more than people wanted. So a subset of
retained mode graphics using draw paths to form the bounds of objects is
where the discussion is currently. Things like font, brush, or other
retained mode features would be left to the canvas context as is.

Rich


Rich Schwerdtfeger
CTO Accessibility Software Group

"Tab Atkins Jr." ---06/23/2011 03:30:32 PM---On Thu, Jun 23, 2011 at 12:56
PM, Richard Schwerdtfeger <schwer@us.ibm.com> wrote:

From: "Tab Atkins Jr." <jackalmage@gmail.com>
To: Richard Schwerdtfeger/Austin/IBM@IBMUS
Cc: Cameron McCormack <cam@mcc.id.au>, Charles Pritchard <chuck@jumis.com>,
Cynthia Shelly <cyns@microsoft.com>, david.bolter@gmail.com, Frank Olivier
<Frank.Olivier@microsoft.com>, Mike@w3.org, public-canvas-api@w3.org,
public-html@w3.org, public-html-a11y@w3.org
Date: 06/23/2011 03:30 PM
Subject: Re: hit testing and retained graphics
________________________________________



On Thu, Jun 23, 2011 at 12:56 PM, Richard Schwerdtfeger
<schwer@us.ibm.com> wrote:
>> So normally, I imagine, hit testing would be done either by using
>> isPointInPath() or by custom code looking at a mouse event’s x/y values.
>> I think this proposal doesn’t work with isPointInPath(), though, is that
>> right?
>>
> I think it would but we would need to incorporate Z order and a notion of
> the last drawn element to compute which element is on top. The user agent
> would need to manage this.

You are attempting to recreate a retained-mode API in an
immediate-mode API. Why is "use SVG" not sufficient for this?

~TJ