This thread is huge - suggested actions (was Re: [Moderator Action] hit testing and retained graphics)

Hi Folks,

This thread has now gotten very long. While much of it is useful discussion, I suspect at this point it is hard for many in the WG to follow, and I also see some points being repeated. At this point, I'd like to make a few suggestions:

(1) When posting on this thread, consider carefully whether you are actually adding genuinely new to the discussion or just repeating points that were already made.
(2) Consider condensing this discussion into specific bug reports.
(3) Consider gathering data on the wiki, for example summarizing key points on the thread, or collecting use cases, or outlining a proposal.

Regards,
Maciej

On Jul 8, 2011, at 3:35 PM, Charles Pritchard wrote:

> On 7/8/2011 2:16 PM, Tab Atkins Jr. wrote:
>> On Fri, Jul 8, 2011 at 1:38 PM, Charles Pritchard<chuck@jumis.com>  wrote:
>>> On 7/8/2011 11:35 AM, Tab Atkins Jr. wrote:
>>>> On Fri, Jul 8, 2011 at 9:08 AM, Richard Schwerdtfeger<schwer@us.ibm.com>
>>>>  wrote:
>>>>> 2. we are roughly 1 feature away from fixing
>>>>> canvas accessibility.
>>>> No we're not.  Not by a long shot.  I've given several reasons why not
>>>> in the past.
>>> Citation please. I haven't seen these -several- reasons that we're far away
>>> from canvas a11y.
>> For one simple example I've brought up several times, and Jonas has
>> brought up in this thread as well, rich text editing.  Trying to do
>> this in Canvas is a horrifying trainwreck of problems, both large and
>> larger, of which dealing with bounding boxes is only one.
> 
> 
> These issues are more related to DOM than Canvas. Hironori Bono has brought up
> IME extensions as well as spell check extensions. Though they did not see acceptance
> in the WHATWG mailing list, he has continued to push forward on them.
> 
> I've not found RTE to be a horrifying train-wreck. We followed the CSS line box specs
> and things worked out.
> 
> I'm aware of the neat issues surrounding this kind of work. Regardless, RTE was
> ruled out as a use case by all major vendors last year. I'd imagine that future
> developments on contentEditable will enable Canvas RTE as they add new vocabulary.
> 
>> It appears that you have to solve all of these problems to solve the
>> "use canvas to do remote access" use-case, though, which is a problem.
> 
> 
> We can only improve remote access, not "solve it".
> 
> Web Apps has a wealth of functionality to help us with remote access,
> from binary data options to dataTransfer semantics.
> 
> VNC clients have proven that a web app can run as well as a desktop app for remote access.
> 
> We now have caret location, we have text selection (via DOM range), binary transfer
> and drag/drop support and focus events. Full screen mouse capture is under discussion,
> headway is being made.
> 
> That's pretty good.
> 
> 
>>>>> The smoothing of shapes has nothing to do with retained mode.
>>>> Um, it has everything to do with retained mode if you want it to be
>>>> automatic.  By definition.
>>> Again, automatic is a strange term that keeps coming up.
>>> In css you can set something to 100%, which makes it "automatic",
>>> whereas with canvas you do need to monitor for resize events
>>> and gather various metrics to have the same result.
>>> 
>>> But gosh that's really common for canvas. It's a low level API.
>> Yes.  My point and Doug's is that you have to do it manually.
>> Retained-mode APIs can do it automatically .
> 
> 
> Yes, in Canvas you have to do things like resolution management
> and repainting in the scripting environment. It's true.
> 
> With the canvas API must create your own scene graph to meet your project needs.
> You're talking about a few lines of code, to manage these "automatic" needs.
> 
> Yes, Canvas is more script heavy than SVG.
> 
> Historically (and currently) scenes can be rendered faster in Canvas
> than in SVG implementations on the mobile and desktop. Currently,
> there is wider support for Canvas-based SVG rendering than UA based
> rendering.
> 
> When it comes to it, I start with Canvas and target SVG second,
> for purposes of compatibility and speed. As for the costs of implementation,
> it really depends on the application.
> 
> SVG FE and VE are not widely supported, which has until WebGL, knocked
> out SVG as an option for some applications.
> 
> SVG scenes require a rigid DOM structure -- so generally, I will use it
> as an output/input/interop format, but I don't work on a live SVG tree,
> as it's much easier to work on strings, arrays and objects in JS than a DOM tree.
> 
> SVG is sorely lacking a replicate tag. With a few tweaks to the spec,
> SVG could have a retained bitmap option, which would help me around most
> of the issues I've encountered. There is an accumulate option for similar performance reasons.
> 
>> Like I've said before (and Jonas has stated in different terms), the
>> problem is not "how do we make canvas accessible?".  It's "how do we
>> solve problem X for users in a way that's accessible?".  The correct
>> solution for "how do I do rich text editing in an accessible way" is
>> "use contenteditable", for example.  If contenteditable isn't subpar,
>> we can fix those problems more easily than we can add sufficient
>> primitives to allow authors to reinvent contenteditable in canvas.
> Jonas said, how do we make application X accessible.
> I believe that's what he said anyway.
> 
> The list has seen many application X examples.
> 
> I'm pretty sure you and Jonas are saying two different things.
> 
> 
>>> As it happens, Canvas is a low level API, and SVG a high level format.
>>> We're trying to fix the low level API before taking on the high level
>>> format.
>> SVG is made out of rectangles, circles, and paths.  You stroke them
>> and fill them.  It's roughly the same level as canvas.  Their only
>> difference is the constantly-mentioned retained vs immediate
>> distinction.
> It's made of complex use tags and transforms, of references and of animations,
> and of css inheritance, of caching and resolution management and additional stroke styles.
> 
> Most of those items are handled by the author in canvas; their management is handled by the implementation
> in SVG. It's a higher level language.
> 
>>> Regardless, I think SVG is wonderful, and I'd rather be spending my
>>> time addressing issues on SVG. But I'm stuck here, in the low level
>>> API, until the foundation is solid.
>>> 
>>> There's a very near and dear issue to me with SVG, and that's
>>> the capability to display InkML rendering. SVG is currently quite poor
>>> at handling pressure values on lines, and using repeated elements
>>> really takes a toll on most implementations.
>> You mean like a single line that gets thinner and thicker at different
>> points?  SVG handles that currently by making the entire stroke a path
>> and filling it.  It would be easier if you could provide a variable
>> stroke width instead, though.
> We're not able to do nice tapering of edges, and the calculations required to make the path,
> as well as to write it out (you have to do a loop, essentially) really are a lot of extra work.
> 
> We lose directional information, which is a shame. I've tried a few tests to see how I can
> replicate things, when it comes to it, I can use InkML and walk the DOM with the scripting
> environment, but I end up with thousands of DOM nodes to get the desired effect.
> 
> Render time is 10x as slow on Mobile Safari with the SVG scene as with the Canvas scene,
> further, any zooming in triggers repainting mechanisms which can't be managed by the author.
> 
> With Canvas, I can manage repainting and bitmap resolution, which is really handy
> for responsiveness.
> 
> 
> 
>>>>> Why have you not pushed to ditch
>>>>> canvas from HTML in favor of SVG?
>>>> Because that would be a stupid thing to push for, frankly.  Doug's a
>>>> smart guy.  He recognizes, like everyone else, that immediate mode
>>>> APIs have an advantage over retained mode APIs in some circumstances.
>>> There's WebGL, a mix of immediate and retained mode API; keep that in mind.
>>> It works because it should. It's closer a format for efficient InkML
>>> processing
>>> than either Canvas or SVG.
>>> 
>>> Again, we're looking at low level API and high level format;
>>> this confusion over retained mode is a red herring.
>> No, it's not.  Several of the problems you've presented are solved
>> *much* more easily if you start with a retained-mode graphics format.
>> That's one of the major points we keep trying to make - you've put
>> blinders on and limited yourself to solving problems using *only* the
>> 2d canvas context.  I guarantee that this will make your solutions
>> much worse than if you try and solve them with the best tool for each
>> job.
> 
> But I've shown you the code that's necessary to solve the problems I've mentioned.
> It takes two additional method calls to bind text to an area on the screen. That's not difficult.
> 
> I've also looked at the code bases for Mozilla and WebKit -- the very basics that I need
> to solve the issue (sending information to the Accessibility Tree) take less than a dozen
> lines of code in each branch.
> 
> The more complex items brought up would take more code, but they're
> certainly no -large- undertaking.
> 
> It can't be *much* more easily when the solution at hand is not that difficult.
> 
> If you want more code samples, to help you understand the proposals,
> I'm willing to do that. I'm willing to point at line numbers in the WebKit and Mozilla
> code bases as well, and demonstrate which function calls are necessary to pass
> information to the accessibility tree.
> 
> I mean, I can work with *easy* as a qualifier. It simply has not been a qualifier here,
> *correctness* has been the qualifier. And by all measure, the *correct* means, in the
> opinion of many vendors, is to use SVG and simply -not- use Canvas.
> 
> -Charles
>

Received on Saturday, 9 July 2011 06:15:56 UTC