Re: Async Image -> ImageData conversion

If it's difficult to make ImageBitmap both an efficient drawing source and
a conversion intermediary, then we could add conversion functions to each
object that can be converted, e.g.:

HTMLImageElement.toBlob()
HTMLImageElement.toImageData()
Blob.toImageData()
ImageData.toBlob()

This seems inconsistent. For example to convert a Blob to an Image, you
should create a URL and set an Image's src to it; to convert a Blob to an
ImageBitmap, you should use createImageBitmap(blob); and then under this
proposal there's a third approach to convert a Blob to an ImageData, using
Blob.toImageData(). It also has a larger surface area, requiring changes to
several interfaces.

So perhaps it would be better to make ImageData the "hub" of conversion,
more like the first proposal I made. If ImageBitmap is a GPU-hosted
premultiplied resource, then ImageData is an expressly not-on-GPU
not-premultiplied alternative. That would mean adding something like this
to ImageData:

typedef (HTMLImageElement or Blob or ImageBitmap) ImageDataSource;

partial interface ImageData {
    static Promise<ImageData> create(ImageDataSource source)
    Promise<Blob> toBlob()
}

ImageData can also be converted to HTMLImageElement via toBlob, ImageBitmap
via createImageBitmap, or a canvas via putImageData (which is still
synchronous, but no longer needs to be done purely for conversion reasons,
so probably means you really want it to appear in the canvas and therefore
should remain synchronous).

This covers all the conversion use cases I can think of without
complicating ImageBitmap's "without undue latency" requirement. There's no
more transferring going on either, which I think is now unnecessary since
you can get from HTMLImageElement to ImageData with one call. I think it's
more future-proof too since future types can be added to ImageDataSource
allowing new types to be converted without a new method being added.

Does this approach sound better?

Ashley


On 26 June 2015 at 16:37, Boris Zbarsky <bzbarsky@mit.edu> wrote:

> On 6/26/15 4:07 AM, Ashley Gullen wrote:
>
>> I don't think we should extend HTMLImageElement because it is not
>> available in workers. Adding the conversion methods to ImageBitmap
>> allows workers to perform conversions using Blob (compressed image data)
>> in the place of HTMLImageElement.
>>
>
> Maybe I wasn't clear.  I was suggesting that we have the methods on both
> HTMLImageElement and ImageBitmap (and possibly on any other things we feel
> should have the methods directly).
>
>  I like the suggestion that "ImageBitmap be the hub of image conversion",
>>
>
> I agree that it sounds appealing, but it means ImageBitmap now has to
> serve two masters: it has to be something that you can paint from quickly
> (premultiplied, probably lives on the GPU) _and_ it needs to be something
> you can transferToImageData efficiently (better to not live on the GPU for
> this).
>
> Maybe that's OK; it's just a bit of a warning flag from my point of view
> when a single object is meant to do multiple quite different things; it
> makes it harder to have it be good at all of them...
>
> -Boris
>