Re: Thoughts on canvas compositing

On Jan 14, 2011, at 2:04 am, Paul Bakaus wrote:

> Hi everyone,
> 
> As you all know, Canvas still has a long way to go until the performance can match classical game/3D environments. This reason largely seems to be related to compositing (please correct me if wrong). Even if the GPU is largely handling all rendering through a 3D context (I.e. WebGL), the render still has to go through the usual mostly CPU intensive path of compositing to overlay or underlay layers of elements.

That's not true of all user agents. For example, WebKit on Mac will use a fast, GPU-based path for compositing a WebGL canvas. It's also able to do this while preserving correct CSS rendering in most cases; for example, you can position other HTML content on top of the WebGL.

> For this particular reason, I am proposing the addition of two features to the way Canvas works (I've included public-fx as this involves more than just the API):
> 
> - Introduce a new mime type for canvas (i.e. application/canvas) and make a canvas self hostable (like SVG, run example.canvas in your browser to get a full screen window wide canvas). This might introduce some challenges, like where the scripting goes. Maybe it still needs to be accompanied with a XML wrapper.

There's a proposed fullscreen API <https://wiki.mozilla.org/Gecko:FullScreenAPI> which could be used to take a canvas element fullscreen. It would be up to the user agent to detect this, and optimize the rendering if possible.

> - Create an attribute similar to wmode or windowless mode in Flash to toggle compositing. By setting the attribute to true (defaults to false), The canvas will become non-transparent and overlay everything else on the page, regardless of the z-index.

Ick!

> The only issue I'm seeing is that the ability to overlay UI controls for various reasons, I.e. Game menus, would not be possible this way. The best possible implementation for us as game developers would be a mix of both – allow overlaying elements on top of the canvas, but simply ignore transparency on element backgrounds that overlap the canvas, which might still improve the compositing.
> 
> I would love to hear thoughts, possibly also from people who are working on compositing directly, to see if my assumptions are correct or if there are other important bottlenecks.

I really don't see the need for any new properties. I think the onus is on the browser developers to detect and implement compositing optimizations when they can.

Simon