Re: Proposed CG consensus position re: Float16Array + HTML Canvas

FYI.

https://github.com/whatwg/html/issues/8708
https://lists.w3.org/Archives/Public/public-html/2023Jan/0002.html

On Mon, Jan 9, 2023 at 8:58 PM Pierre-Anthony Lemieux <pal@sandflow.com> wrote:
>
> Good morning/evening,
>
> Having not heard objections, I plan to share the input below on JAN 10
> with the WHATWG HTML workstream and the W3C HTML WG.
>
> Let me know if it should go elsewhere.
>
> Best,
>
> -- Pierre
>
> On Mon, Dec 26, 2022 at 11:04 AM Pierre-Anthony Lemieux
> <pal@sandflow.com> wrote:
> >
> > Good morning/evening,
> >
> > Per our last call, please review the proposed CG consensus position
> > below and provide your input no later than January 8 COB. Thanks to
> > Ken Russell for doing the bulk of the work.
> >
> > The intent is to communicate the position to other groups, within and
> > outside the W3C.
> >
> > Best,
> >
> > -- CG chairs
> >
> > # CG proposal re: Float16Array + HTML Canvas
> >
> > The ColorWeb CG is seeking consensus with parties interested in
> > discussing floating-point backing stores for canvas rendering
> > contexts.
> >
> > The CG spec proposal is:
> > https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md
> >
> > The primary outstanding issue at present is how to represent values
> > which are read back from the canvas into typed arrays which can be
> > manipulated from ECMAScript.
> >
> > The current spec proposal supports readbacks into both
> > Uint8ClampedArray and Float32Array via ImageDataColorType.
> >
> > It has been suggested to also add support for readbacks into
> > Float16Array (https://github.com/w3c/ColorWeb-CG/issues/87). This
> > would make the specification dependent on the Float16Array type
> > currently being defined by the ECMAScript committee (ISO/TC 39).
> >
> > The CG supports the idea of adding support for Float16Array to
> > ECMAScript. However, the CG feels it is most appropriate to decouple
> > the ongoing improvements to ECMAScript from the work on floating-point
> > canvases. Users of the web platform are expressing the need for
> > floating-point canvas backing stores now; in Chromium, implementation
> > is underway in crbug.com/1230619. Moreover, readback and CPU-side
> > manipulation paths are not high-performance, so the additional memory
> > bandwidth of using Float32Array for this task will not critically
> > impact applications.
> >
> > The ColorWeb CG therefore suggests:
> >
> > 1) Moving ahead with
> > https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md in its
> > current form, with unorm8 and float32 as options for readback.
> >
> > 2) In parallel, working on adding Float16Array to ECMAScript, and
> > prototyping it in browsers.
> >
> > 3) Specifying a "float16" enum in ImageDataColorType once
> > implementation experience has been achieved with Float16Array, .
> > Applications can feature-detect its support in browsers by catching
> > exceptions raised by getImageData or createImageData.
> >
> > # Background
> >
> > For context, introducing a Float16Array into the Typed Array hierarchy
> > was discussed several years ago among members of TC39, as well as the
> > Khronos Group where Typed Arrays originated. At the time, the costs of
> > supporting this type natively in ECMAScript engines outweighed the
> > benefits for the following reasons:
> >
> > 1) For practical purposes, Float16Array could be sufficiently
> > polyfilled in ECMAScript:
> >
> > https://github.com/petamoriken/float16
> >
> > Corner cases existed regarding denormalized values, NaNs, and
> > infinities - but applications could generally achieve their desired
> > results, with good performance on all ECMAScript engines, using
> > existing primitives.
> >
> > 2) Direct CPU support for half-float numbers did not seem to exist. At
> > the time, C libraries which worked with this data type universally
> > emulated it, rather than using compiler intrinsics on certain
> > platforms.
> >
> > 3) A significant amount of work would be required in every ECMAScript
> > engine to make this typed array type perform well.
> >
> > 4) A significant amount of work would be required to specify the
> > behavior of this numeric type in ECMAScript.
> >
> > The landscape has changed in recent years. Per
> > https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Hardware_support,
> > ARM and x86 processors either support FP16 natively, or will in the
> > near future. FP16 formats are increasingly heavily used in neural
> > network evaluation and image processing, including on the web. The
> > ECMAScript editors may be able to save spec work by referencing IEEE
> > specifications (which, to be fair, already existed when Float16Array
> > was originally considered).

Received on Tuesday, 10 January 2023 22:42:56 UTC