SVG for Canvas a11y - was Re: hit testing and retained graphics

All,

I do not believe that a retained-mode API has been requested -- in a 
retained mode API, it would be possible
for the developer to retain path data.

This is not necessary to accomplish the task at hand; we only need to send
the path data to the UA, after that, there's no further exposure to the 
author.

Using SVG is a bit -heavy- when addressing the basic use case.

SVG has getAttribute and setAttribute, always allowing the scripting 
environment
to access the path information of elements.

That's not necessary to address the issue we're having. In many ways, 
it's far more
than we are looking for. Using SVG as the -only- option would be more 
costly
in terms of memory and cpu usage, and likely more costly on developers.

I would certainly like to see a concrete SVG proposal, as I would like 
to see
SVG and Canvas interoperate on a closer level. I do not think that such 
a proposal
is mutually exclusive of adding a new method to the canvas 2d context
to inform the accessibility tree / assisitve technology of the current path.

Is there room for an SVG+Canvas profile to be developed, and to have a 
simple
implementation of setElementPath?

Because, the latter will take a minimum of effort to implement, and the 
former
would be very welcome, but will take some time to roll out. Both 
solutions are usable
and valid, I'd like to see them both developed.

-Charles

On 7/11/2011 2:37 PM, Richard Schwerdtfeger wrote:
>
> James,
>
> Here is the link to Doug's proposal. There is more on his blog 
> referenced in the post: 
> http://lists.w3.org/Archives/Public/public-canvas-api/2011AprJun/0117.html
>
> I am willing to work to develop it more with Doug - if the browser 
> manufacturers agree this is the right approach. Unfortunately, I don't 
> have the luxury of developing different approaches up to see if Jell-O 
> sticks to the wall with people. I would like to see if people on the 
> list, especially browser manufacturers, think this is the right approach.
>
> At this point it needs more development. The benefit of this approach 
> is that it makes use of the fallback content of canvas to drive the 
> platform accessibility API as does the rest of canvas with the added 
> benefit of using SVG drawing paths to supply the bounds of objects. To 
> me there are a lot of questions to be worked out like what do we do 
> with the existing SVG keyboard navigation and how would this effect 
> the browsers layout engine but those are details to be worked out. I 
> just need to know that this is the preferred direction to head in.
>
> Rich
>
>
> Rich Schwerdtfeger
> CTO Accessibility Software Group
>
> Inactive hide details for James Robinson ---07/11/2011 04:22:34 
> PM---On Mon, Jul 11, 2011 at 2:19 PM, Richard Schwerdtfeger <scJames 
> Robinson ---07/11/2011 04:22:34 PM---On Mon, Jul 11, 2011 at 2:19 PM, 
> Richard Schwerdtfeger <schwer@us.ibm.com>wrote: >
>
> From: James Robinson <jamesr@google.com>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS
> Cc: "Tab Atkins Jr." <jackalmage@gmail.com>, Charles McCathieNevile 
> <chaals@opera.com>, Charles Pritchard <chuck@jumis.com>, Henri Sivonen 
> <hsivonen@iki.fi>, Karl Dubost <karld@opera.com>, 
> "public-canvas-api@w3.org" <public-canvas-api@w3.org>, 
> "public-html@w3.org" <public-html@w3.org>, "public-html-a11y@w3.org" 
> <public-html-a11y@w3.org>, public-html-request@w3.org, Doug Schepers 
> <schepers@w3.org>
> Date: 07/11/2011 04:22 PM
> Subject: Re: hit testing and retained graphics
>
> ------------------------------------------------------------------------
>
>
>
>
>
> On Mon, Jul 11, 2011 at 2:19 PM, Richard Schwerdtfeger 
> <_schwer@us.ibm.com_ <mailto:schwer@us.ibm.com>> wrote:
>
>
>
>       Rich Schwerdtfeger
>       CTO Accessibility Software Group
>
>       _public-html-request@w3.org_
>       <mailto:public-html-request@w3.org>wrote on 07/08/2011 04:16:38 PM:
>
>       > From: "Tab Atkins Jr." <_jackalmage@gmail.com_
>       <mailto:jackalmage@gmail.com>>
>       > To: Charles Pritchard <_chuck@jumis.com_ <mailto:chuck@jumis.com>>
>       > Cc: Richard Schwerdtfeger/Austin/IBM@IBMUS, Doug Schepers
>       > <_schepers@w3.org_ <mailto:schepers@w3.org>>, Charles
>       McCathieNevile <_chaals@opera.com_ <mailto:chaals@opera.com>>,
>       Henri
>       > Sivonen <_hsivonen@iki.fi_ <mailto:hsivonen@iki.fi>>, Karl
>       Dubost <_karld@opera.com_ <mailto:karld@opera.com>>, "public-
>
>       > _canvas-api@w3.org_ <mailto:canvas-api@w3.org>"
>       <_public-canvas-api@w3.org_ <mailto:public-canvas-api@w3.org>>,
>       "_public-html@w3.org_ <mailto:public-html@w3.org>"
>       > <_public-html@w3.org_ <mailto:public-html@w3.org>>,
>       "_public-html-a11y@w3.org_ <mailto:public-html-a11y@w3.org>"
>       <public-html-
>       > _a11y@w3.org_ <mailto:a11y@w3.org>>,
>       _public-html-request@w3.org_ <mailto:public-html-request@w3.org>
>       > Date: 07/08/2011 04:18 PM
>
>       > Subject: Re: hit testing and retained graphics
>       > Sent by: _public-html-request@w3.org_
>       <mailto:public-html-request@w3.org>
>       >
>       > On Fri, Jul 8, 2011 at 1:38 PM, Charles Pritchard
>       <_chuck@jumis.com_ <mailto:chuck@jumis.com>> wrote:
>       > > On 7/8/2011 11:35 AM, Tab Atkins Jr. wrote:
>       > >> On Fri, Jul 8, 2011 at 9:08 AM, Richard
>       Schwerdtfeger<_schwer@us.ibm.com_ <mailto:schwer@us.ibm.com>>
>       > >>  wrote:
>       > >>> 2. we are roughly 1 feature away from fixing
>       > >>> canvas accessibility.
>       > >>
>       > >> No we're not.  Not by a long shot.  I've given several
>       reasons why not
>       > >> in the past.
>       > >
>       > > Citation please. I haven't seen these -several- reasons that
>       we're far away
>       > > from canvas a11y.
>       > >
>       > > A11y itself is a huge field ready for innovation; we're not
>       able to come to
>       > > those innovations while we're still fighting for the basics,
>       like bounding
>       > > boxes.
>       > >
>       > > Canvas a11y has been under discussion since 2009, these
>       issues have not
>       > > been shifting... they're the same ones as they were back
>       then, for the same
>       > > set of software and tools there were back then.
>       > >
>       > > We've addressed most of the issues, Richard has personally
>       spoken
>       > > with several AT vendors about it, and I've done the code
>       > > and AT testing across Windows, OS X and Mobile Safari.
>       > >
>       > > These are reality-based actions. I'm fine exploring
>       theoretic problems,
>       > > I'm happy to do that, but I'd like there to be a distinction
>       and recognition
>       > > of the work that has been done.
>       >
>       > For one simple example I've brought up several times, and
>       Jonas has
>       > brought up in this thread as well, rich text editing. Trying to do
>       > this in Canvas is a horrifying trainwreck of problems, both
>       large and
>       > larger, of which dealing with bounding boxes is only one.
>       >
>
>       The subject is hit testing and retained graphics. We need this
>       regardless of the rich text editing discussion. We are losing
>       focus on the problem.
>
>       I think we spend enough cycles on the rich text editing discussion.
>
>
>       > It appears that you have to solve all of these problems to
>       solve the
>       > "use canvas to do remote access" use-case, though, which is a
>       problem.
>       >
>       >
>       > >>> The smoothing of shapes has nothing to do with retained mode.
>       > >>
>       > >> Um, it has everything to do with retained mode if you want
>       it to be
>       > >> automatic.  By definition.
>       > >
>       > > Again, automatic is a strange term that keeps coming up.
>       > > In css you can set something to 100%, which makes it
>       "automatic",
>       > > whereas with canvas you do need to monitor for resize events
>       > > and gather various metrics to have the same result.
>       > >
>       > > But gosh that's really common for canvas. It's a low level API.
>       >
>       > Yes. My point and Doug's is that you have to do it manually.
>       > Retained-mode APIs can do it automatically .
>       >
>       >
>       > >>> What we are
>       > >>> trying to do is make canvas accessible. SVG is another
>       technology and
>       > >>> that
>       > >>> is a separate accessibility effort.
>       > >>
>       > >> Conway's Law strikes again.
>       > >
>       > > This is a subtle put down; I get it, but it's absurd.
>       > >
>       > > You're telling us that, because our teams are using canvas,
>       and have come to
>       > > us with issues regarding canvas, that we've become deluded.
>       >
>       > I wouldn't say "deluded", but I would say that you're focused
>       in an
>       > unproductive way.
>       >
>       > Like I've said before (and Jonas has stated in different
>       terms), the
>       > problem is not "how do we make canvas accessible?". It's "how
>       do we
>       > solve problem X for users in a way that's accessible?". The
>       correct
>       > solution for "how do I do rich text editing in an accessible
>       way" is
>       > "use contenteditable", for example. If contenteditable isn't
>       subpar,
>       > we can fix those problems more easily than we can add sufficient
>       > primitives to allow authors to reinvent contenteditable in canvas.
>       >
>       We are getting off track with contenteditable. This discussion
>       is simply about providing the AT the location and bounds of an
>       object.
>
>       >
>       > > As it happens, Canvas is a low level API, and SVG a high
>       level format.
>       > > We're trying to fix the low level API before taking on the
>       high level
>       > > format.
>       >
>       > SVG is made out of rectangles, circles, and paths. You stroke them
>       > and fill them. It's roughly the same level as canvas. Their only
>       > difference is the constantly-mentioned retained vs immediate
>       > distinction.
>       >
>       > (SVG does include some higher-level constructs like filters, but
>       > that's irrelevant to the discussion at hand.)
>       >
>       >
>       OK. ... What do you think of Doug Schepers proposal to have an
>       API that would integrate SVG into Canvas such that the retained
>       mode aspects are limited to SVG?
>
>
>       > >>> If SVG is so wonderful why are you not
>       > >>> pushing for an SVG element in HTML 5?
>       > >>
>       > >> SVG is in HTML.<_http://whatwg.org/html#svg-0_>
>       > >
>       > > More appropriate link:
>       > > _http://www.w3.org/TR/html5/the-map-element.html#svg-0_
>       > >
>       > > SVG is mentioned in HTML. Processing is not specified in
>       that document.
>       > > Object tags are mentioned in HTML as well, that doesn't mean
>       that Java
>       > > is part of HTML5.
>       >
>       > I was flippantly answering what I thought was a rude and
>       unproductive
>       > question. Don't worry too much over the details. ^_^
>       >
>       >
>       > > Regardless, I think SVG is wonderful, and I'd rather be
>       spending my
>       > > time addressing issues on SVG. But I'm stuck here, in the
>       low level
>       > > API, until the foundation is solid.
>       > >
>       > > There's a very near and dear issue to me with SVG, and that's
>       > > the capability to display InkML rendering. SVG is currently
>       quite poor
>       > > at handling pressure values on lines, and using repeated
>       elements
>       > > really takes a toll on most implementations.
>       >
>       > You mean like a single line that gets thinner and thicker at
>       different
>       > points? SVG handles that currently by making the entire stroke
>       a path
>       > and filling it. It would be easier if you could provide a variable
>       > stroke width instead, though.
>       >
>       >
>       > >>> Why have you not pushed to ditch
>       > >>> canvas from HTML in favor of SVG?
>       > >>
>       > >> Because that would be a stupid thing to push for, frankly.
>        Doug's a
>       > >> smart guy.  He recognizes, like everyone else, that
>       immediate mode
>       > >> APIs have an advantage over retained mode APIs in some
>       circumstances.
>       > >
>       > > There's WebGL, a mix of immediate and retained mode API;
>       keep that in mind.
>       > > It works because it should. It's closer a format for
>       efficient InkML
>       > > processing
>       > > than either Canvas or SVG.
>       > >
>       > > Again, we're looking at low level API and high level format;
>       > > this confusion over retained mode is a red herring.
>       >
>       > No, it's not. Several of the problems you've presented are solved
>       > *much* more easily if you start with a retained-mode graphics
>       format.
>       > That's one of the major points we keep trying to make - you've put
>       > blinders on and limited yourself to solving problems using
>       *only* the
>       > 2d canvas context. I guarantee that this will make your solutions
>       > much worse than if you try and solve them with the best tool
>       for each
>       > job.
>       >
>       The problem is authors don't always agree with what we might
>       think is the best tool for the job. I would really like to hear
>       you weigh in on Doug Schepers proposal. We can't leave canvas
>       inaccessible. Authors will use canvas to draw objects on the
>       screen and some will want what they produce to be accessible. We
>       need to provide them the tools to support interoperability with
>       assistive technologies. They need to provide the bounds of an
>       object to meet that requirement.
>
>       Would what Doug is suggesting, using SVG as Doug suggested be an
>       acceptable solution to Webkit and Chrome? 
>
>
> That's impossible to evaluate without a concrete proposal.
>
> - James
>
>
>
>       > ~TJ
>       > 
>
>