Re: hit testing and retained graphics

On Mon, Jul 11, 2011 at 2:19 PM, Richard Schwerdtfeger <schwer@us.ibm.com>wrote:

>
>
> Rich Schwerdtfeger
> CTO Accessibility Software Group
>
> public-html-request@w3.org wrote on 07/08/2011 04:16:38 PM:
>
> > From: "Tab Atkins Jr." <jackalmage@gmail.com>
> > To: Charles Pritchard <chuck@jumis.com>
> > Cc: Richard Schwerdtfeger/Austin/IBM@IBMUS, Doug Schepers
> > <schepers@w3.org>, Charles McCathieNevile <chaals@opera.com>, Henri
> > Sivonen <hsivonen@iki.fi>, Karl Dubost <karld@opera.com>, "public-
>
> > canvas-api@w3.org" <public-canvas-api@w3.org>, "public-html@w3.org"
> > <public-html@w3.org>, "public-html-a11y@w3.org" <public-html-
> > a11y@w3.org>, public-html-request@w3.org
> > Date: 07/08/2011 04:18 PM
>
> > Subject: Re: hit testing and retained graphics
> > Sent by: public-html-request@w3.org
> >
> > On Fri, Jul 8, 2011 at 1:38 PM, Charles Pritchard <chuck@jumis.com>
> wrote:
> > > On 7/8/2011 11:35 AM, Tab Atkins Jr. wrote:
> > >> On Fri, Jul 8, 2011 at 9:08 AM, Richard Schwerdtfeger<
> schwer@us.ibm.com>
> > >>  wrote:
> > >>> 2. we are roughly 1 feature away from fixing
> > >>> canvas accessibility.
> > >>
> > >> No we're not.  Not by a long shot.  I've given several reasons why not
> > >> in the past.
> > >
> > > Citation please. I haven't seen these -several- reasons that we're far
> away
> > > from canvas a11y.
> > >
> > > A11y itself is a huge field ready for innovation; we're not able to
> come to
> > > those innovations while we're still fighting for the basics, like
> bounding
> > > boxes.
> > >
> > > Canvas a11y has been under discussion since 2009, these issues have not
> > > been shifting... they're the same ones as they were back then, for the
> same
> > > set of software and tools there were back then.
> > >
> > > We've addressed most of the issues, Richard has personally spoken
> > > with several AT vendors about it, and I've done the code
> > > and AT testing across Windows, OS X and Mobile Safari.
> > >
> > > These are reality-based actions. I'm fine exploring theoretic problems,
> > > I'm happy to do that, but I'd like there to be a distinction and
> recognition
> > > of the work that has been done.
> >
> > For one simple example I've brought up several times, and Jonas has
> > brought up in this thread as well, rich text editing. Trying to do
> > this in Canvas is a horrifying trainwreck of problems, both large and
> > larger, of which dealing with bounding boxes is only one.
> >
>
> The subject is hit testing and retained graphics. We need this regardless
> of the rich text editing discussion. We are losing focus on the problem.
>
> I think we spend enough cycles on the rich text editing discussion.
>
>
> > It appears that you have to solve all of these problems to solve the
> > "use canvas to do remote access" use-case, though, which is a problem.
> >
> >
> > >>> The smoothing of shapes has nothing to do with retained mode.
> > >>
> > >> Um, it has everything to do with retained mode if you want it to be
> > >> automatic.  By definition.
> > >
> > > Again, automatic is a strange term that keeps coming up.
> > > In css you can set something to 100%, which makes it "automatic",
> > > whereas with canvas you do need to monitor for resize events
> > > and gather various metrics to have the same result.
> > >
> > > But gosh that's really common for canvas. It's a low level API.
> >
> > Yes. My point and Doug's is that you have to do it manually.
> > Retained-mode APIs can do it automatically .
> >
> >
> > >>> What we are
> > >>> trying to do is make canvas accessible. SVG is another technology and
> > >>> that
> > >>> is a separate accessibility effort.
> > >>
> > >> Conway's Law strikes again.
> > >
> > > This is a subtle put down; I get it, but it's absurd.
> > >
> > > You're telling us that, because our teams are using canvas, and have
> come to
> > > us with issues regarding canvas, that we've become deluded.
> >
> > I wouldn't say "deluded", but I would say that you're focused in an
> > unproductive way.
> >
> > Like I've said before (and Jonas has stated in different terms), the
> > problem is not "how do we make canvas accessible?". It's "how do we
> > solve problem X for users in a way that's accessible?". The correct
> > solution for "how do I do rich text editing in an accessible way" is
> > "use contenteditable", for example. If contenteditable isn't subpar,
> > we can fix those problems more easily than we can add sufficient
> > primitives to allow authors to reinvent contenteditable in canvas.
> >
> We are getting off track with contenteditable. This discussion is simply
> about providing the AT the location and bounds of an object.
>
> >
> > > As it happens, Canvas is a low level API, and SVG a high level format.
> > > We're trying to fix the low level API before taking on the high level
> > > format.
> >
> > SVG is made out of rectangles, circles, and paths. You stroke them
> > and fill them. It's roughly the same level as canvas. Their only
> > difference is the constantly-mentioned retained vs immediate
> > distinction.
> >
> > (SVG does include some higher-level constructs like filters, but
> > that's irrelevant to the discussion at hand.)
> >
> >
> OK. ... What do you think of Doug Schepers proposal to have an API that
> would integrate SVG into Canvas such that the retained mode aspects are
> limited to SVG?
>
>
> > >>> If SVG is so wonderful why are you not
> > >>> pushing for an SVG element in HTML 5?
> > >>
> > >> SVG is in HTML.<http://whatwg.org/html#svg-0>
> > >
> > > More appropriate link:
> > > http://www.w3.org/TR/html5/the-map-element.html#svg-0
> > >
> > > SVG is mentioned in HTML. Processing is not specified in that document.
> > > Object tags are mentioned in HTML as well, that doesn't mean that Java
> > > is part of HTML5.
> >
> > I was flippantly answering what I thought was a rude and unproductive
> > question. Don't worry too much over the details. ^_^
> >
> >
> > > Regardless, I think SVG is wonderful, and I'd rather be spending my
> > > time addressing issues on SVG. But I'm stuck here, in the low level
> > > API, until the foundation is solid.
> > >
> > > There's a very near and dear issue to me with SVG, and that's
> > > the capability to display InkML rendering. SVG is currently quite poor
> > > at handling pressure values on lines, and using repeated elements
> > > really takes a toll on most implementations.
> >
> > You mean like a single line that gets thinner and thicker at different
> > points? SVG handles that currently by making the entire stroke a path
> > and filling it. It would be easier if you could provide a variable
> > stroke width instead, though.
> >
> >
> > >>> Why have you not pushed to ditch
> > >>> canvas from HTML in favor of SVG?
> > >>
> > >> Because that would be a stupid thing to push for, frankly.  Doug's a
> > >> smart guy.  He recognizes, like everyone else, that immediate mode
> > >> APIs have an advantage over retained mode APIs in some circumstances.
> > >
> > > There's WebGL, a mix of immediate and retained mode API; keep that in
> mind.
> > > It works because it should. It's closer a format for efficient InkML
> > > processing
> > > than either Canvas or SVG.
> > >
> > > Again, we're looking at low level API and high level format;
> > > this confusion over retained mode is a red herring.
> >
> > No, it's not. Several of the problems you've presented are solved
> > *much* more easily if you start with a retained-mode graphics format.
> > That's one of the major points we keep trying to make - you've put
> > blinders on and limited yourself to solving problems using *only* the
> > 2d canvas context. I guarantee that this will make your solutions
> > much worse than if you try and solve them with the best tool for each
> > job.
> >
> The problem is authors don't always agree with what we might think is the
> best tool for the job. I would really like to hear you weigh in on Doug
> Schepers proposal. We can't leave canvas inaccessible. Authors will use
> canvas to draw objects on the screen and some will want what they produce to
> be accessible. We need to provide them the tools to support interoperability
> with assistive technologies. They need to provide the bounds of an object to
> meet that requirement.
>
> Would what Doug is suggesting, using SVG as Doug suggested be an acceptable
> solution to Webkit and Chrome?
>

That's impossible to evaluate without a concrete proposal.

- James


>
>
> > ~TJ
> >
>
>