Re: hit testing and retained graphics

Rich Schwerdtfeger
CTO Accessibility Software Group

public-html-request@w3.org wrote on 07/08/2011 04:16:38 PM:

> From: "Tab Atkins Jr." <jackalmage@gmail.com>
> To: Charles Pritchard <chuck@jumis.com>
> Cc: Richard Schwerdtfeger/Austin/IBM@IBMUS, Doug Schepers
> <schepers@w3.org>, Charles McCathieNevile <chaals@opera.com>, Henri
> Sivonen <hsivonen@iki.fi>, Karl Dubost <karld@opera.com>, "public-
> canvas-api@w3.org" <public-canvas-api@w3.org>, "public-html@w3.org"
> <public-html@w3.org>, "public-html-a11y@w3.org" <public-html-
> a11y@w3.org>, public-html-request@w3.org
> Date: 07/08/2011 04:18 PM
> Subject: Re: hit testing and retained graphics
> Sent by: public-html-request@w3.org
>
> On Fri, Jul 8, 2011 at 1:38 PM, Charles Pritchard <chuck@jumis.com>
wrote:
> > On 7/8/2011 11:35 AM, Tab Atkins Jr. wrote:
> >> On Fri, Jul 8, 2011 at 9:08 AM, Richard
Schwerdtfeger<schwer@us.ibm.com>
> >>  wrote:
> >>> 2. we are roughly 1 feature away from fixing
> >>> canvas accessibility.
> >>
> >> No we're not.  Not by a long shot.  I've given several reasons why not
> >> in the past.
> >
> > Citation please. I haven't seen these -several- reasons that we're far
away
> > from canvas a11y.
> >
> > A11y itself is a huge field ready for innovation; we're not able to
come to
> > those innovations while we're still fighting for the basics, like
bounding
> > boxes.
> >
> > Canvas a11y has been under discussion since 2009, these issues have not
> > been shifting... they're the same ones as they were back then, for the
same
> > set of software and tools there were back then.
> >
> > We've addressed most of the issues, Richard has personally spoken
> > with several AT vendors about it, and I've done the code
> > and AT testing across Windows, OS X and Mobile Safari.
> >
> > These are reality-based actions. I'm fine exploring theoretic problems,
> > I'm happy to do that, but I'd like there to be a distinction and
recognition
> > of the work that has been done.
>
> For one simple example I've brought up several times, and Jonas has
> brought up in this thread as well, rich text editing.  Trying to do
> this in Canvas is a horrifying trainwreck of problems, both large and
> larger, of which dealing with bounding boxes is only one.
>
The subject is hit testing and retained graphics. We need this regardless
of the rich text editing discussion. We are losing focus on the problem.

I think we spend enough cycles on the rich text editing discussion.

> It appears that you have to solve all of these problems to solve the
> "use canvas to do remote access" use-case, though, which is a problem.
>
>
> >>> The smoothing of shapes has nothing to do with retained mode.
> >>
> >> Um, it has everything to do with retained mode if you want it to be
> >> automatic.  By definition.
> >
> > Again, automatic is a strange term that keeps coming up.
> > In css you can set something to 100%, which makes it "automatic",
> > whereas with canvas you do need to monitor for resize events
> > and gather various metrics to have the same result.
> >
> > But gosh that's really common for canvas. It's a low level API.
>
> Yes.  My point and Doug's is that you have to do it manually.
> Retained-mode APIs can do it automatically .
>
>
> >>> What we are
> >>> trying to do is make canvas accessible. SVG is another technology and
> >>> that
> >>> is a separate accessibility effort.
> >>
> >> Conway's Law strikes again.
> >
> > This is a subtle put down; I get it, but it's absurd.
> >
> > You're telling us that, because our teams are using canvas, and have
come to
> > us with issues regarding canvas, that we've become deluded.
>
> I wouldn't say "deluded", but I would say that you're focused in an
> unproductive way.
>
> Like I've said before (and Jonas has stated in different terms), the
> problem is not "how do we make canvas accessible?".  It's "how do we
> solve problem X for users in a way that's accessible?".  The correct
> solution for "how do I do rich text editing in an accessible way" is
> "use contenteditable", for example.  If contenteditable isn't subpar,
> we can fix those problems more easily than we can add sufficient
> primitives to allow authors to reinvent contenteditable in canvas.
>
We are getting off track with contenteditable. This discussion is simply
about providing the AT the location and bounds of an object.

>
> > As it happens, Canvas is a low level API, and SVG a high level format.
> > We're trying to fix the low level API before taking on the high level
> > format.
>
> SVG is made out of rectangles, circles, and paths.  You stroke them
> and fill them.  It's roughly the same level as canvas.  Their only
> difference is the constantly-mentioned retained vs immediate
> distinction.
>
> (SVG does include some higher-level constructs like filters, but
> that's irrelevant to the discussion at hand.)
>
>
OK. ... What do you think of Doug Schepers proposal to have an API that
would integrate SVG into Canvas such that the retained mode aspects are
limited to SVG?

> >>> If SVG is so wonderful why are you not
> >>> pushing for an SVG element in HTML 5?
> >>
> >> SVG is in HTML.<http://whatwg.org/html#svg-0>
> >
> > More appropriate link:
> > http://www.w3.org/TR/html5/the-map-element.html#svg-0
> >
> > SVG is mentioned in HTML. Processing is not specified in that document.
> > Object tags are mentioned in HTML as well, that doesn't mean that Java
> > is part of HTML5.
>
> I was flippantly answering what I thought was a rude and unproductive
> question.  Don't worry too much over the details.  ^_^
>
>
> > Regardless, I think SVG is wonderful, and I'd rather be spending my
> > time addressing issues on SVG. But I'm stuck here, in the low level
> > API, until the foundation is solid.
> >
> > There's a very near and dear issue to me with SVG, and that's
> > the capability to display InkML rendering. SVG is currently quite poor
> > at handling pressure values on lines, and using repeated elements
> > really takes a toll on most implementations.
>
> You mean like a single line that gets thinner and thicker at different
> points?  SVG handles that currently by making the entire stroke a path
> and filling it.  It would be easier if you could provide a variable
> stroke width instead, though.
>
>
> >>> Why have you not pushed to ditch
> >>> canvas from HTML in favor of SVG?
> >>
> >> Because that would be a stupid thing to push for, frankly.  Doug's a
> >> smart guy.  He recognizes, like everyone else, that immediate mode
> >> APIs have an advantage over retained mode APIs in some circumstances.
> >
> > There's WebGL, a mix of immediate and retained mode API; keep that in
mind.
> > It works because it should. It's closer a format for efficient InkML
> > processing
> > than either Canvas or SVG.
> >
> > Again, we're looking at low level API and high level format;
> > this confusion over retained mode is a red herring.
>
> No, it's not.  Several of the problems you've presented are solved
> *much* more easily if you start with a retained-mode graphics format.
> That's one of the major points we keep trying to make - you've put
> blinders on and limited yourself to solving problems using *only* the
> 2d canvas context.  I guarantee that this will make your solutions
> much worse than if you try and solve them with the best tool for each
> job.
>
The problem is authors don't always agree with what we might think is the
best tool for the job. I would really like to hear you weigh in on Doug
Schepers proposal. We can't leave canvas inaccessible. Authors will use
canvas to draw objects on the screen and some will want what they produce
to be accessible. We need to provide them the tools to support
interoperability with assistive technologies. They need to provide the
bounds of an object to meet that requirement.

Would what Doug is suggesting, using SVG as Doug suggested be an acceptable
solution to Webkit and Chrome?

> ~TJ
>