Re: Proposed CG consensus position re: Float16Array + HTML Canvas

Good morning/evening,

Having not heard objections, I plan to share the input below on JAN 10
with the WHATWG HTML workstream and the W3C HTML WG.

Let me know if it should go elsewhere.

Best,

-- Pierre

On Mon, Dec 26, 2022 at 11:04 AM Pierre-Anthony Lemieux
<pal@sandflow.com> wrote:
>
> Good morning/evening,
>
> Per our last call, please review the proposed CG consensus position
> below and provide your input no later than January 8 COB. Thanks to
> Ken Russell for doing the bulk of the work.
>
> The intent is to communicate the position to other groups, within and
> outside the W3C.
>
> Best,
>
> -- CG chairs
>
> # CG proposal re: Float16Array + HTML Canvas
>
> The ColorWeb CG is seeking consensus with parties interested in
> discussing floating-point backing stores for canvas rendering
> contexts.
>
> The CG spec proposal is:
> https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md
>
> The primary outstanding issue at present is how to represent values
> which are read back from the canvas into typed arrays which can be
> manipulated from ECMAScript.
>
> The current spec proposal supports readbacks into both
> Uint8ClampedArray and Float32Array via ImageDataColorType.
>
> It has been suggested to also add support for readbacks into
> Float16Array (https://github.com/w3c/ColorWeb-CG/issues/87). This
> would make the specification dependent on the Float16Array type
> currently being defined by the ECMAScript committee (ISO/TC 39).
>
> The CG supports the idea of adding support for Float16Array to
> ECMAScript. However, the CG feels it is most appropriate to decouple
> the ongoing improvements to ECMAScript from the work on floating-point
> canvases. Users of the web platform are expressing the need for
> floating-point canvas backing stores now; in Chromium, implementation
> is underway in crbug.com/1230619. Moreover, readback and CPU-side
> manipulation paths are not high-performance, so the additional memory
> bandwidth of using Float32Array for this task will not critically
> impact applications.
>
> The ColorWeb CG therefore suggests:
>
> 1) Moving ahead with
> https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md in its
> current form, with unorm8 and float32 as options for readback.
>
> 2) In parallel, working on adding Float16Array to ECMAScript, and
> prototyping it in browsers.
>
> 3) Specifying a "float16" enum in ImageDataColorType once
> implementation experience has been achieved with Float16Array, .
> Applications can feature-detect its support in browsers by catching
> exceptions raised by getImageData or createImageData.
>
> # Background
>
> For context, introducing a Float16Array into the Typed Array hierarchy
> was discussed several years ago among members of TC39, as well as the
> Khronos Group where Typed Arrays originated. At the time, the costs of
> supporting this type natively in ECMAScript engines outweighed the
> benefits for the following reasons:
>
> 1) For practical purposes, Float16Array could be sufficiently
> polyfilled in ECMAScript:
>
> https://github.com/petamoriken/float16
>
> Corner cases existed regarding denormalized values, NaNs, and
> infinities - but applications could generally achieve their desired
> results, with good performance on all ECMAScript engines, using
> existing primitives.
>
> 2) Direct CPU support for half-float numbers did not seem to exist. At
> the time, C libraries which worked with this data type universally
> emulated it, rather than using compiler intrinsics on certain
> platforms.
>
> 3) A significant amount of work would be required in every ECMAScript
> engine to make this typed array type perform well.
>
> 4) A significant amount of work would be required to specify the
> behavior of this numeric type in ECMAScript.
>
> The landscape has changed in recent years. Per
> https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Hardware_support,
> ARM and x86 processors either support FP16 natively, or will in the
> near future. FP16 formats are increasingly heavily used in neural
> network evaluation and image processing, including on the web. The
> ECMAScript editors may be able to save spec work by referencing IEEE
> specifications (which, to be fair, already existed when Float16Array
> was originally considered).

Received on Tuesday, 10 January 2023 04:58:56 UTC