Re: Async Image -> ImageData conversion

On Wed, Jun 24, 2015 at 1:28 PM, Ashley Gullen <ashley@scirra.com> wrote:
> Sorry for the confusion. Yes, the latest URL is:
> https://www.scirra.com/labs/specs/imagebitmap-conversion-extensions.html
> I'm new to specs and WebIDL, my intent was to say those are new methods on
> ImageBitmap. Is "partial interface ImageBitmap" the correct way to say that?
> (I updated the URL to say that instead)
>
> The wording of "undue latency" in the ImageBitmap spec does not appear to
> rule out ImageBitmap storing its encoded image data, and lazily decoding it.
> However I'm not sure what the point of that would be, since it seems to be
> how drawing HTMLImageElements to a canvas already works. I see a couple of
> options:
> - allow lazy decoding in ImageBitmap, and therefore converting to ImageData
> is an explicit request to decompress
> - require ImageBitmap to decompress its contents, and make it available
> through a read-only Uint8ClampedArray like ImageData has. Therefore,
> converting to ImageData indicates the intent to modify this content,
> therefore a copy is warranted.
> - provide a way to "neuter" the ImageBitmap when converting it to ImageData,
> transferring its contents and avoiding any copy, which is useful if the
> ImageBitmap is a temporary object only being created for the purposes of
> getting the ImageData. I guess this would be similar to transferrable
> objects with workers.

Perhaps this could be done by specifying a "transferToImageData" method.

This proposal makes ImageBitmap neuterable, and there's intent to
implement it in order to allow rendering to canvas contexts from
worker threads: https://wiki.whatwg.org/wiki/OffscreenCanvas .

-Ken


> - add some kind of "load" method on ImageBitmap. It may store its contents
> in compressed form, but calling "load" causes it to be decompressed (or
> loaded from somewhere else). The ImageBitmap is only guaranteed to be drawn
> with "undue latency" after the "load" call. Therefore for the use case of
> low-latency rendering "load" can always be called immediately after
> creation, but for conversion purposes not calling "load" avoids duplicating
> memory.
>
> Note for devices with discrete GPUs, it's possible that ImageBitmap stores
> the decompressed image data in a GPU texture but does not have it in system
> RAM. In this case making a copy for the ImageData is also warranted.
>
> I'm a web developer proposing this because it would be useful for my
> purposes, I've no idea which of these would be favoured by implementors or
> the spec process. I'm happy to get involved in spec work though so please
> let me know if you have any feedback/suggestions on anything I'm doing here.
>
> Ashley
>
>
>
> On 24 June 2015 at 19:12, Boris Zbarsky <bzbarsky@mit.edu> wrote:
>>
>> On 6/19/15 5:43 AM, Ashley Gullen wrote:
>>>
>>> I've not done this before, so I've no idea if this is the right/useful
>>> approach, but I drafted a spec for it here:
>>>
>>> https://www.scirra.com/labs/specs/imagedata-blob-extensions.html
>>
>>
>> Ashley,
>>
>> We at Mozilla were just discussing this proposal, and we have a concern.
>>
>> With this proposal, going from an <img> to an ImageData requires
>> conversion of an intermediate ImageBitmap.  Unfortunately, an ImageBitmap is
>> specified to be paintable "without undue latency", which we interpret to
>> mean it needs to have decoded image data, and the ImageData will end up
>> needing to copy said data in practice (because it needs an editable copy).
>>
>> That creates two copies of the decoded image data in memory, which seems
>> fairly undesirable on memory-constrained devices.
>>
>> -Boris
>>
>>
>

Received on Wednesday, 24 June 2015 20:54:47 UTC