Re: exposing CANVAS or something like it to Web Workers

On May 14, 2012, at 5:03 PM, Gregg Tavares (勤) <gman@google.com> wrote:

> 
> 
> On Mon, May 14, 2012 at 4:42 PM, Jonas Sicking <jonas@sicking.cc> wrote:
> On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <glenn@zewt.org> wrote:
> > On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <gman@google.com> wrote:
> >
> >> I'd like to work on exposing something like CANVAS to web workers.
> >>
> >> Ideally how over it works I'd like to be able to
> >>
> >> *) get a 2d context in a web worker
> >
> >
> > I'd recommend not trying to tackle 2d and 3d contexts at once, and only
> > worrying about WebGL to start.
> >
> > Another issue: rendering in a worker thread onto a canvas which is displayed
> > in the main thread.  This needs to be solved in a way that doesn't cause the
> > asynchronous nature of what's happening to be visible to scripts.  toDataURL
> > and toBlob would probably need to be prohibited on the canvas element.  I'm
> > not sure what the actual API would look like.
> 
> If/when we do this, I think it should be done in such a way that the
> main window can't access the canvas object at all. Similar to what
> happens when an ArrayBuffer is transferred to a Worker using
> structured cloning. Once a canvas is transferred to a Worker, any
> access to it should throw or return null/0/"". If you want to transfer
> pixel data to the main thread, it seems less racy to do that by
> getting the pixel data in the Worker which owns the canvas and then
> transfer that to the main thread using postMessage.
> 
> How about separating the canvasy parts of canvas from canvas and the imagy parts of image from image.
> 
> In other words, Imagine canvas is implemented like this
> 
> class Canvas : public HTMLElement {
>   private:
>     CanvasSurface* m_surface;  // everything about canvas that is not HTMLElement
> };
> 
> And that Image is similarly implemented as
> 
> class Image : public HTMLElement {
>   private:
>     Picture* m_picture;  // everything about Image that is not HTMLElement
> }
> 
> now imagine you can instantiate inner implementation of these things. The parts that are not HTMLElement
> 
> var canvasSurface = new CanvasSurface();
> var ctx = canvasSurface.getContext("2d");
> var pic = new Picture;
> pic.src = "http://someplace.com/someimage.jpg";
> pic.onload = function() {
>    ctx.drawImage(pic, 0, );
> }
> 
> Let's assume you can instantiate these things in either the page or a worker. Nether can be transfered.
> 
> Would that work? What problems would that have?
> 

Seems fine. If it works with XHR I'd imagine it'll work with Picture.

Blob uri behavior may come up as an issue. I believe they should "just work". SVG and animated gif would render the same as it would in an Image that has not been added to the dom or is otherwise display: none.

I suspect we may have blob Uris corresponding to live video streams. If those come about and work with onload I'd expect them to work here (simply grabbing the last/first available frame).






>  
> 
> > This would also require some
> > equivalent to requestAnimationFrame in the worker thread.
> 
> Agreed!
> 
> / Jonas
>