Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 6:23 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> On Mon, May 14, 2012 at 5:10 PM, Charles Pritchard <chuck@jumis.com>
> wrote:
> > On May 14, 2012, at 4:42 PM, Jonas Sicking <jonas@sicking.cc> wrote:
> >
> >> On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard <glenn@zewt.org> wrote:
> >>> On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) <gman@google.com>
> wrote:
> >>>
> >>>> I'd like to work on exposing something like CANVAS to web workers.
> >>>>
> >>>> Ideally how over it works I'd like to be able to
> >>>>
> >>>> *) get a 2d context in a web worker
> >>>
> >>>
> >>> I'd recommend not trying to tackle 2d and 3d contexts at once, and only
> >>> worrying about WebGL to start.
> >>>
> >>> Another issue: rendering in a worker thread onto a canvas which is
> displayed
> >>> in the main thread.  This needs to be solved in a way that doesn't
> cause the
> >>> asynchronous nature of what's happening to be visible to scripts.
>  toDataURL
> >>> and toBlob would probably need to be prohibited on the canvas element.
>  I'm
> >>> not sure what the actual API would look like.
> >>
> >> If/when we do this, I think it should be done in such a way that the
> >> main window can't access the canvas object at all. Similar to what
> >> happens when an ArrayBuffer is transferred to a Worker using
> >> structured cloning. Once a canvas is transferred to a Worker, any
> >> access to it should throw or return null/0/"". If you want to transfer
> >> pixel data to the main thread, it seems less racy to do that by
> >> getting the pixel data in the Worker which owns the canvas and then
> >> transfer that to the main thread using postMessage.
> >>
> >>> This would also require some
> >>> equivalent to requestAnimationFrame in the worker thread.
> >>
> >> Agreed!
> >>
> >> / Jonas
> >
> >
> >
> > I'm a bit lost-- wouldn't we just postMessage from the document over to
> the web worker when we want a refresh?
> >
> > I agree that we ought to be transferring pixel data not Canvas contexts;
> with the possible exception of CSS context.
> >
> > We could just create new canvas instances inside the worker thread. I'd
> still prefer to clone CanvasPattern as a means of transferring paint over
> to the worker, though sending pixel data would work too.
> >
> > I heard Picture come up-- it seems like that object might have
> additional semantics for high resolution alternatives that may need to be
> considered.
>
> I was saying that I think we should transfer the context from the main
> thread to the worker thread. The worker thread should then be able to
> use the context to draw directly to the screen without interacting
> with the main thread.
>

That's a great goal but is transferring the right way to do it?

context.canvas is currently a reference to the canvas from which the
context as created. Will that just go NULL on transfer? Will it change to
some other object that is not a Canvas but acts similarly?

What happens to the context on the main page? All of it's methods become
no-ops and it's properties become non-live?

How will things synchronize if I call canvas.toDataURL after passing the
context to a work? Exception? nothing? It seems hard to define how that
would synchronize

If there was a way to allow a worker to generate a media stream and one of
those ways involves an in worker canvas/canvassurface, then it seems like
none of the above questions have to be answered. Instead the worker creates
the surface, and then through some other API attached it to a video tag.
You can call video.toDataURL, that's well defined (I assume)

Synchronization is also somewhat defined in that direction. Doesn't that
seem like the path of least resistance?





>
> You should generally not need to transfer pixel data. But I think it
> should be possible to grab pixel data on whichever thread is currently
> owning a context. This will implicitly make it possible to transfer
> pixel data if anyone wants to do it.
>
> / Jonas
>