Re: [whatwg] Canvas image to blob/dataurl within Worker

Sorry for arriving so late to the party...

I would like to update this thread on what has been happening around
canvas.toBlob in Blink.

First of all, there was an attempt at implementing canvas.toBlob a couple
years ago, back when Chromium was using WebKit. The experience revealed
that the API was not the silver bullet we hoped and was not satisfactorily
addressing issues regarding latency, memory usage and jank reduction.
There are many technical reasons for this, some of which are related to
architectural details that are specific to Chromium (native thread model,
out of process blob storage, etc.)  Long story short, the feature was put
on ice because it failed to provide a significant advantage over solutions
that can be implemented using existing APIs.

Recently, there has been a flare-up of discussion around the fact that
canvas.toBlob is still not shipping in Chrome.  So we came clean about our
intention to not ship it, and we've asked about people's use cases in order
to look for a better solution. It turns out the use cases are quite varied.

For the past week Noel Gordon (the Chromium engineer who was originally
working on toBlob) and I have been brainstorming on some ideas. We noticed
this one on the Mozilla issue tracker (
https://bugzilla.mozilla.org/show_bug.cgi?id=870024) about adding an
ImageData.toBlob, and we also got some ideas from Roc's WorkerCanvas
proposal.  We are still collecting use cases and ideas, on top of the ones
that were already posted on our issue tracker and the blink-dev mailing
list. If anyone reading this wants to add theirs, please use this form:
https://docs.google.com/forms/d/1YHIGf2-0wqbeFLA8Y6S-YaFzCWxlw9xPrOn9ctP5JwA/viewform
We
will hash it all and report back to this thread in a few days with some
data and some ideas.

One idea we've been brewing that has not surfaced on this thread so far is
that of having an ImageBitmap.toBlob(), which would return a Promise. Also,
ImageBitmap should be transferable. This idea is just partially baked right
now, but I wanted to throw it out into the open to get people thinking
about it. The rationale behind this is that ImageBitmap is basically a
lightweight read-only proxy for pretty much anything that can be used as an
image source. So this single API entry-point can perform as well as a
direct toBlob() method hanging off of any other type of object that an
ImageBitmap can be created from. Because it is an opaque and immutable
object, it gives browser vendors a lot of latitude for implementing high
performance code paths. It allows pixel data from the DOM to cross Worker
boundaries without making copies (at least in cases where the source is a
static image). It allows interactions with blob storage to be driven from a
worker (vs. canvas.toBlob), hence avoiding jank on the main thread. Raw
image data generated in JS would not have to transit through a canvas (just
create an ImageBitmap from an ImageData object), which eliminates a lot of
unnecessary pixel pushing. Eliminating the transit through canvas also
allows for fast paths for capturing blobs from <video>, <img>, svg, URLs,
to name a few.

    -Justin

On Wed, Mar 25, 2015 at 6:41 AM, Anne van Kesteren <annevk@annevk.nl> wrote:

> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan
> <robert@ocallahan.org> wrote:
> > My understanding is that the current consensus proposal for canvas in
> > Workers is not what's in the spec, but this:
> > https://wiki.whatwg.org/wiki/WorkerCanvas
> > See "Canvas in Workers" threads from October 2013 for the discussion. svn
> > is failing me but the CanvasProxy proposal in the spec definitely
> predates
> > those threads.
> >
> > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy
> stuff
> > from the spec to avoid confusion.
> >
> > That proposal contains WorkerCanvas.toBlob, which needs to be updated to
> > use promises.
>
> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It
> would be interesting to know what the latest on this is. Canvas in
> workers has been discussed for a long time now and we're still not at
> the point of implementing.
>
>
> --
> https://annevankesteren.nl/
>

Received on Wednesday, 25 March 2015 19:41:06 UTC