Re: correct and incorrect uses of canvas

On 7/16/2011 10:42 AM, Andrew Fedoniouk wrote:
>> -----Original Message----- From: Henri Sivonen
>> Sent: Friday, July 15, 2011 8:11 AM
>> To: Jonas Sicking
>> Cc: Steve Faulkner ; HTMLWG WG ; Frank Olivier ; Richard Schwerdtfeger ;
>> Cynthia Shelly ; David Singer ; Tab Atkins Jr. ; Edward O'Connor ; 
>> Canvas
>> Subject: Re: correct and incorrect uses of canvas
>>
>> On Thu, 2011-07-14 at 13:40 -0700, Jonas Sicking wrote:
>>> We can and should improve the
>>> SVG DOM to attract more people, but I suspect there will always be a
>>> group of people that choose to use canvas anyway due to better
>>> performance or other reasons.
>>
>> Do you have examples of people choosing <canvas> over SVG for perf
>> reasons? Are the perf reasons data-based or merely assumed? Are the perf
>> reasons the kind of quality of implementation issues that can be
>> considered to be transient and addressed over the next couple of years?
>
> Educated and commonsense (I hope) reasoning:
>
> Each DOM element with associated structures is about 200-5000 bytes in
> browser memory (YMMV). Let's consider for example SVG line 
> (x1,y1-x2,y2), that will
> take 2k of RAM (DOM memory).
> In reality you will need around 48 bytes for data structures to 
> represent the
> line in memory. Usually it is stack allocated data so reusable - takes 
> zero memory
> between draw calls.
>
> Imagine that your task is to draw 100 lines randomly.
>
> In SVG you will need to create DOM structure that will hold 200k of 
> memory
> when you actually don't need all those DOM elements at all.
> Just to handle passively those 200k / 100 elements you will need 
> additional
> CPU power - memory management, GC, swapping, etc. Without doing anything
> good at all. Even for people with disabilities - for them it is enough 
> to pronounce
> "100 lines replaced randomly" for example.

While reflecting on why SVG performance was unworkable for some targets
I came across David Dailey's replicate proposal for SVG:

http://srufaculty.sru.edu/david.dailey/svg/SVGOpen2010/replicate.htm

I've opened a few threads on it in the past:
http://lists.w3.org/Archives/Public/www-svg/2010Jun/0049.html
http://lists.w3.org/Archives/Public/www-svg/2011May/0057.html


I'd like to see replicate/useAt cover more use cases;
I've had a strong focus on rendering InkML trace groups.

InkML can reasonably encode the 100 random lines into one trace group,
and it can be easily rendered from that in Canvas. It can be translated 
into SVG,
but performance does suffer until something like replicate is added.

For the time being, this does look like a reasonable use of canvas 
inside of SVG,
as using the live SVG DOM to implement replicate is a bit nasty.

We've been able to use pure SVG or SVG+WebGL filters to replicate the output
of our canvas projects, but performance and compatibility are not 
-there- yet. They've certainly
improved over the past 6 months.

I've been trying to maintain the SVG anyway, as even with a bad 
performance profile,
it is an open w3c standard and an interoperable file format. Mostly, I 
mix InkML+SVG.

InkML with SVG references maps quite well to WebGL.

> Value of SVG a11y in this case is zero if not even less - any 
> accessible tool
> is literally dying on large numbers of DOM elements.

Accessibility tools do not need to peek at each DOM element.
Yes, 100 random line path+styles is still a rather inefficient use of a DOM,
but a11y often has its own tree and can otherwise use simple dom to stay 
away from live areas.

In the case of 100 random lines, with svg, you'd likely mark that group
as aria-hidden and/or give it a presentational role.

Use a rectangle as a bounding box, giving that rectangle a title and 
description.

In high contrast mode you may want the random lines to be disabled; less 
clutter on the screen,
if they're used as a background / for presentational purposes. You'd use 
something like
the css style background: url(#randomlinesgroupid)

High contrast mode disables css background images in Windows FF and IE.

>
>>
>> In theory, SVG should be able to outperform <canvas> for painting,
>> because the browser engine gets to control how often to repaint, what
>> parts to repaint and can avoid intermediate bitmaps when the path
>> stroking and filling can be performed nearer hardware and there are
>> guaranteed not to be readbacks because the browser knows there aren't
>> filters in use.
>
> Actually even in theory it is quite opposite.
>
> Thought experiment:
>
> Let's consider the same 100 lines image with a circle in the middle
> (red button/disk on randomly generated background).
>
<snip>
> In SVG you will need to check 101 elements - complexity is O(n)
> in general.
>
> With our custom solution the complexity is O(1)  - we just need
> to check if point is inside the circle.


You could accomplish the same in SVG by generating a single <path> tag
based on those elements, which receives clicks but is transparent.

It requires additional effort on the programmer,
but I think it would handle the situation. It certainly is less 
efficient than canvas
and would have lower performance, but not -as- bad as hit testing 101 
elements.

>
>>
>> So in theory, if SVG has performance issues, they should be attributable
>> to the DOM. If a sub-DOM is used for accessibility in the <canvas> case,
>> then the <canvas> case has a DOM, too.
>>
>
> SVG has performance issues by its generic nature.
>
> Yes, SVG has DOM. But that does not mean it has reasonable a11y value -
> any given SVG element have significantly less semantics in it than
> any HTML one.

WAI-ARIA roles allow authors to further mark up SVG element semantics.
There's no big difference between <input type="button" /> and <circle 
role="button" />

> I believe that decision to make SVG DOM to be a part of HTML DOM is
> conceptually wrong - too different things. Crossing hedgehog and a snake
> will give you piece of barbed wire but not an animal.

It's useful to be able to embed any XML content into documents, be
it MathML, SVG, InkML or well, anything. Data uris are great as well.

This helps documents to be portable and loss-less.

> I also think that <canvas> as a dedicated DOM element is also a bad idea
> - no semantic and a11y values indeed.

We've enabled the shadow dom, allowing the creation of XBL-style components.
<canvas> complex shadow dom tree </canvas> works as a nice standalone 
component.

Canvas does require a coordinate space, much as SVG does;
it's important that I can run: <canvas role="application" width="200" 
style="width: 400px">

Those are some reasons for a dedicated canvas tag. The img tag has
a lot of compatibility / baggage attached, and so the new canvas tag 
with similar structure,
makes sense.

> In my opinion SVG should not be different from PNG, GIF, EMF and other
> image formats - SVG images should be used as backgrounds or <img 
> src=".svg">
>
> Instead of <canvas> we just need mechanism that allows to synthesize
> images on client side (Image.getGraphics())  and use them as any other
> image in HTML/CSS - in CSS and in <img>.

Both of these exist in Mozilla and WebKit families. IE has granted this 
kind of access to
css images via Active X since IE6, if not earlier.

> In this case developers will be forced to use HTML elements with 
> proper semantic.
> And so the scene with random lines and button above will be defined as:
>
> <div style="background: url(in-memory:100-random-lines)">
> <button id="ignition" class="big-red-button"></button>
> </div>

All of these seem fine to me, though my pseudo-code may be a little off:

Canvas:
<canvas><button id="ignition" class="big-red-button"></button></canvas>

HTML+CSS:
<div style="background: url(#svg-Or-Canvas-With-100-Random-lines)">
<button id="ignition" class="big-red-button"></button>
</div>

SVG:
<svg><rect fill="#random-lines" aria-owns="ignition" 
/><foreignObject><button id="ignition" 
class="big-red-button"></button></svg>

They're "compatible" with HTML 3.2, they can be progressively enhanced
and the button semantic is maintained for compatibility with older AT 
software.

-Charles

Received on Saturday, 16 July 2011 19:14:23 UTC