Re: Async Image -> ImageData conversion

I see that OffscreenCanvas also specifies a close() method for ImageBitmap.
Perhaps browsers could use copy-on-write with the toImageData() method?
Then you can efficiently either transfer or copy depending on if you close
the ImageBitmap you transferred to an ImageData.

Transfer:
createImageBitmap() -> imageBitmap.toImageData() -> imageBitmap.close()
No copy made, contents transferred to the imageData

createImageBitmap() -> imageBitmap.toImageData() -> access resulting
imageData.data
Copy made on first access



On 24 June 2015 at 21:54, Kenneth Russell <kbr@google.com> wrote:

> On Wed, Jun 24, 2015 at 1:28 PM, Ashley Gullen <ashley@scirra.com> wrote:
> > Sorry for the confusion. Yes, the latest URL is:
> > https://www.scirra.com/labs/specs/imagebitmap-conversion-extensions.html
> > I'm new to specs and WebIDL, my intent was to say those are new methods
> on
> > ImageBitmap. Is "partial interface ImageBitmap" the correct way to say
> that?
> > (I updated the URL to say that instead)
> >
> > The wording of "undue latency" in the ImageBitmap spec does not appear to
> > rule out ImageBitmap storing its encoded image data, and lazily decoding
> it.
> > However I'm not sure what the point of that would be, since it seems to
> be
> > how drawing HTMLImageElements to a canvas already works. I see a couple
> of
> > options:
> > - allow lazy decoding in ImageBitmap, and therefore converting to
> ImageData
> > is an explicit request to decompress
> > - require ImageBitmap to decompress its contents, and make it available
> > through a read-only Uint8ClampedArray like ImageData has. Therefore,
> > converting to ImageData indicates the intent to modify this content,
> > therefore a copy is warranted.
> > - provide a way to "neuter" the ImageBitmap when converting it to
> ImageData,
> > transferring its contents and avoiding any copy, which is useful if the
> > ImageBitmap is a temporary object only being created for the purposes of
> > getting the ImageData. I guess this would be similar to transferrable
> > objects with workers.
>
> Perhaps this could be done by specifying a "transferToImageData" method.
>
> This proposal makes ImageBitmap neuterable, and there's intent to
> implement it in order to allow rendering to canvas contexts from
> worker threads: https://wiki.whatwg.org/wiki/OffscreenCanvas .
>
> -Ken
>
>
> > - add some kind of "load" method on ImageBitmap. It may store its
> contents
> > in compressed form, but calling "load" causes it to be decompressed (or
> > loaded from somewhere else). The ImageBitmap is only guaranteed to be
> drawn
> > with "undue latency" after the "load" call. Therefore for the use case of
> > low-latency rendering "load" can always be called immediately after
> > creation, but for conversion purposes not calling "load" avoids
> duplicating
> > memory.
> >
> > Note for devices with discrete GPUs, it's possible that ImageBitmap
> stores
> > the decompressed image data in a GPU texture but does not have it in
> system
> > RAM. In this case making a copy for the ImageData is also warranted.
> >
> > I'm a web developer proposing this because it would be useful for my
> > purposes, I've no idea which of these would be favoured by implementors
> or
> > the spec process. I'm happy to get involved in spec work though so please
> > let me know if you have any feedback/suggestions on anything I'm doing
> here.
> >
> > Ashley
> >
> >
> >
> > On 24 June 2015 at 19:12, Boris Zbarsky <bzbarsky@mit.edu> wrote:
> >>
> >> On 6/19/15 5:43 AM, Ashley Gullen wrote:
> >>>
> >>> I've not done this before, so I've no idea if this is the right/useful
> >>> approach, but I drafted a spec for it here:
> >>>
> >>> https://www.scirra.com/labs/specs/imagedata-blob-extensions.html
> >>
> >>
> >> Ashley,
> >>
> >> We at Mozilla were just discussing this proposal, and we have a concern.
> >>
> >> With this proposal, going from an <img> to an ImageData requires
> >> conversion of an intermediate ImageBitmap.  Unfortunately, an
> ImageBitmap is
> >> specified to be paintable "without undue latency", which we interpret to
> >> mean it needs to have decoded image data, and the ImageData will end up
> >> needing to copy said data in practice (because it needs an editable
> copy).
> >>
> >> That creates two copies of the decoded image data in memory, which seems
> >> fairly undesirable on memory-constrained devices.
> >>
> >> -Boris
> >>
> >>
> >
>