Re: Proposed CG consensus position re: Float16Array + HTML Canvas

Thank you for writing this up. I have updated
https://github.com/WebKit/standards-positions/issues/91 with a link to this.





On Tue, Jan 10, 2023 at 11:43 PM Pierre-Anthony Lemieux <pal@sandflow.com>
wrote:

> FYI.
>
> https://github.com/whatwg/html/issues/8708
> https://lists.w3.org/Archives/Public/public-html/2023Jan/0002.html
>
> On Mon, Jan 9, 2023 at 8:58 PM Pierre-Anthony Lemieux <pal@sandflow.com>
> wrote:
> >
> > Good morning/evening,
> >
> > Having not heard objections, I plan to share the input below on JAN 10
> > with the WHATWG HTML workstream and the W3C HTML WG.
> >
> > Let me know if it should go elsewhere.
> >
> > Best,
> >
> > -- Pierre
> >
> > On Mon, Dec 26, 2022 at 11:04 AM Pierre-Anthony Lemieux
> > <pal@sandflow.com> wrote:
> > >
> > > Good morning/evening,
> > >
> > > Per our last call, please review the proposed CG consensus position
> > > below and provide your input no later than January 8 COB. Thanks to
> > > Ken Russell for doing the bulk of the work.
> > >
> > > The intent is to communicate the position to other groups, within and
> > > outside the W3C.
> > >
> > > Best,
> > >
> > > -- CG chairs
> > >
> > > # CG proposal re: Float16Array + HTML Canvas
> > >
> > > The ColorWeb CG is seeking consensus with parties interested in
> > > discussing floating-point backing stores for canvas rendering
> > > contexts.
> > >
> > > The CG spec proposal is:
> > > https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md
> > >
> > > The primary outstanding issue at present is how to represent values
> > > which are read back from the canvas into typed arrays which can be
> > > manipulated from ECMAScript.
> > >
> > > The current spec proposal supports readbacks into both
> > > Uint8ClampedArray and Float32Array via ImageDataColorType.
> > >
> > > It has been suggested to also add support for readbacks into
> > > Float16Array (https://github.com/w3c/ColorWeb-CG/issues/87). This
> > > would make the specification dependent on the Float16Array type
> > > currently being defined by the ECMAScript committee (ISO/TC 39).
> > >
> > > The CG supports the idea of adding support for Float16Array to
> > > ECMAScript. However, the CG feels it is most appropriate to decouple
> > > the ongoing improvements to ECMAScript from the work on floating-point
> > > canvases. Users of the web platform are expressing the need for
> > > floating-point canvas backing stores now; in Chromium, implementation
> > > is underway in crbug.com/1230619. Moreover, readback and CPU-side
> > > manipulation paths are not high-performance, so the additional memory
> > > bandwidth of using Float32Array for this task will not critically
> > > impact applications.
> > >
> > > The ColorWeb CG therefore suggests:
> > >
> > > 1) Moving ahead with
> > > https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md in its
> > > current form, with unorm8 and float32 as options for readback.
> > >
> > > 2) In parallel, working on adding Float16Array to ECMAScript, and
> > > prototyping it in browsers.
> > >
> > > 3) Specifying a "float16" enum in ImageDataColorType once
> > > implementation experience has been achieved with Float16Array, .
> > > Applications can feature-detect its support in browsers by catching
> > > exceptions raised by getImageData or createImageData.
> > >
> > > # Background
> > >
> > > For context, introducing a Float16Array into the Typed Array hierarchy
> > > was discussed several years ago among members of TC39, as well as the
> > > Khronos Group where Typed Arrays originated. At the time, the costs of
> > > supporting this type natively in ECMAScript engines outweighed the
> > > benefits for the following reasons:
> > >
> > > 1) For practical purposes, Float16Array could be sufficiently
> > > polyfilled in ECMAScript:
> > >
> > > https://github.com/petamoriken/float16
> > >
> > > Corner cases existed regarding denormalized values, NaNs, and
> > > infinities - but applications could generally achieve their desired
> > > results, with good performance on all ECMAScript engines, using
> > > existing primitives.
> > >
> > > 2) Direct CPU support for half-float numbers did not seem to exist. At
> > > the time, C libraries which worked with this data type universally
> > > emulated it, rather than using compiler intrinsics on certain
> > > platforms.
> > >
> > > 3) A significant amount of work would be required in every ECMAScript
> > > engine to make this typed array type perform well.
> > >
> > > 4) A significant amount of work would be required to specify the
> > > behavior of this numeric type in ECMAScript.
> > >
> > > The landscape has changed in recent years. Per
> > >
> https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Hardware_support
> ,
> > > ARM and x86 processors either support FP16 natively, or will in the
> > > near future. FP16 formats are increasingly heavily used in neural
> > > network evaluation and image processing, including on the web. The
> > > ECMAScript editors may be able to save spec work by referencing IEEE
> > > specifications (which, to be fair, already existed when Float16Array
> > > was originally considered).
>
>