Re: Async Image -> ImageData conversion

Making ImageData the "hub" would imply more copies in memory in many cases.
Because ImageData is mutable (not to mention the alpha multiplication
issues which are also a factor), it cannot share its image buffer with the
source it was created from, unlike ImageBitmap.  Immutability is a
significant advantage of ImageBitmap, which allows for zero-copy code paths
in many cases, which helps with both performance and memory consumption.

On Tue, Jun 30, 2015 at 4:17 PM, Ashley Gullen <ashley@scirra.com> wrote:

> If it's difficult to make ImageBitmap both an efficient drawing source and
> a conversion intermediary, then we could add conversion functions to each
> object that can be converted, e.g.:
>
> HTMLImageElement.toBlob()
> HTMLImageElement.toImageData()
> Blob.toImageData()
> ImageData.toBlob()
>
> This seems inconsistent. For example to convert a Blob to an Image, you
> should create a URL and set an Image's src to it; to convert a Blob to an
> ImageBitmap, you should use createImageBitmap(blob); and then under this
> proposal there's a third approach to convert a Blob to an ImageData, using
> Blob.toImageData(). It also has a larger surface area, requiring changes to
> several interfaces.
>
> So perhaps it would be better to make ImageData the "hub" of conversion,
> more like the first proposal I made. If ImageBitmap is a GPU-hosted
> premultiplied resource, then ImageData is an expressly not-on-GPU
> not-premultiplied alternative. That would mean adding something like this
> to ImageData:
>
> typedef (HTMLImageElement or Blob or ImageBitmap) ImageDataSource;
>
> partial interface ImageData {
>     static Promise<ImageData> create(ImageDataSource source)
>     Promise<Blob> toBlob()
> }
>
> ImageData can also be converted to HTMLImageElement via toBlob,
> ImageBitmap via createImageBitmap, or a canvas via putImageData (which is
> still synchronous, but no longer needs to be done purely for conversion
> reasons, so probably means you really want it to appear in the canvas and
> therefore should remain synchronous).
>
> This covers all the conversion use cases I can think of without
> complicating ImageBitmap's "without undue latency" requirement. There's no
> more transferring going on either, which I think is now unnecessary since
> you can get from HTMLImageElement to ImageData with one call. I think it's
> more future-proof too since future types can be added to ImageDataSource
> allowing new types to be converted without a new method being added.
>
> Does this approach sound better?
>
> Ashley
>
>
> On 26 June 2015 at 16:37, Boris Zbarsky <bzbarsky@mit.edu> wrote:
>
>> On 6/26/15 4:07 AM, Ashley Gullen wrote:
>>
>>> I don't think we should extend HTMLImageElement because it is not
>>> available in workers. Adding the conversion methods to ImageBitmap
>>> allows workers to perform conversions using Blob (compressed image data)
>>> in the place of HTMLImageElement.
>>>
>>
>> Maybe I wasn't clear.  I was suggesting that we have the methods on both
>> HTMLImageElement and ImageBitmap (and possibly on any other things we feel
>> should have the methods directly).
>>
>>  I like the suggestion that "ImageBitmap be the hub of image conversion",
>>>
>>
>> I agree that it sounds appealing, but it means ImageBitmap now has to
>> serve two masters: it has to be something that you can paint from quickly
>> (premultiplied, probably lives on the GPU) _and_ it needs to be something
>> you can transferToImageData efficiently (better to not live on the GPU for
>> this).
>>
>> Maybe that's OK; it's just a bit of a warning flag from my point of view
>> when a single object is meant to do multiple quite different things; it
>> makes it harder to have it be good at all of them...
>>
>> -Boris
>>
>
>