Re: [w3ctag/design-reviews] Review OffscreenCanvas, including ImageBitmapRenderingContext (#141)

Hi sorry about the slow response time.  

Regarding the need for both ImageData and ImageBitmap:
>From a pure API perspective, I agree that ImageData alone could be sufficient.  The motivation for ImageBitmap is not to add functionality to the platform, it is purely a performance primitive.  Because the object is opaque and read-only,  implementations can do a lot of very significant optimizations. For example:
* The pixel data is not required to reside in RAM. It may reside on the GPU, thus sparing costly GPU<->CPU transfers. With ImageData, this is not the case becase the pixel data must be mapped to a script-accessible typed ArrayBuffer.
* Taking a snapshot of a canvas as an ImageBitmap does not require making an upfront copy of the pixel data. Instead, the ImageBitmap object can refer to the canvas's backing store until the time that the canvas's contents are modified (we've been calling this trick "copy on write").  This optimization saves memory as well as CPU time
* No need for converting the pixel data.  Because ImageData strictly requires RGBA layout, 8-bits per component and color values not premultiplied by alpha, there are usually conversions that are necessary in copying data to and from an ImageData.  ImageBitmap can skip conversions by keeping the data in whatever format is convenient to the implementation.
* Asynchronous creation. Making image decoding, resizing and reformatting asynchronous (and therefore parallelizable) is useful for making apps that run smoother.

Regarding the ImageBitmapRenderingContext: it exists for the same reason as ImageBitmap.  It adds no functionality to the platform. We could get by with using CanvasRenderingContext2D.drawImage to display an ImageBitmap. However, the 2D context requires making an additional copy of the image (to the canvas backing store).  ImageBitmapRenderingContext provides transfer semantics, which avoid the data duplication (saves CPU time as well as RAM).

The use of transfer semantics is an answer to web developer requests for zero-copy mechanisms for handling pixel data. We've been told, for example, that it is extremely challenging to make mobile web apps that manipulates DSLR-resolution images because of memory bloat with the current APIs

Here is a talk I gave a couple weeks ago that may help put things in perspective it is mostly about ImageBitmap and OffscreenCanvas and it provides a bit of use case context: https://www.youtube.com/watch?v=wkDd-x0EkFU


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/141#issuecomment-306887913