Re: Linear canvas vs non-linear pixels.

Chris – can you also comment (if you know) about mobile OS support?  For example, does the MacOS state also match iOS?   What about Android?   What about performance concerns of floating point operations and lack of GPU on many lower end Android devices?

Leonard

From: Christopher Cameron <ccameron@google.com>
Date: Wednesday, March 17, 2021 at 2:40 AM
To: Pierre-Anthony Lemieux <pal@sandflow.com>
Cc: "public-colorweb@w3.org" <public-colorweb@w3.org>
Subject: Re: Linear canvas vs non-linear pixels.
Resent-From: <public-colorweb@w3.org>
Resent-Date: Wednesday, March 17, 2021 at 2:39 AM



On Tue, Mar 16, 2021 at 1:50 PM Pierre-Anthony Lemieux <pal@sandflow.com<mailto:pal@sandflow.com>> wrote:
Thanks to Chris for drafting a concrete proposal [0]. Below is a
potential challenge when handling PQ HDR content.

Starting from mastering, video pixels are coded as non-linear integer
code values, e.g. HLG, PQ, etc..

These pixels ideally remain unmodified until they are rendered to the
display since each conversion carries a risk that fidelity will be
lost. This is the case even in the case where the color space and the
pixel encoded remain the same.

The strawman currently requires that PQ pixels be converted to linear
values when written to Canvas:

   PQ CV ----F----> linear value -----G-----> PQ CV

This is non-trivial and computationally expensive since F is highly
non-linear and covers many orders of magnitude. It also requires G to
be the exact inverse of F, which is not always simple if F and G are
provided by different implementations.

How do we provide a means of avoiding this complexity and potential
loss of fidelity?

Could one approach be to add optional support for non-linear pixel
encodings, e.g. add 'rec2100--pq' to [1]. In other words,
implementation would need to support 'rec2020-linear' but could also
support 'rec2100--pq'.

Adding a 'rec2100-pq' and 'rec2100-hlg' color is quite reasonable.

Support is spotty on desktop operating systems.

  *   Windows has a "rec2100-pq + rgb10-alpha2" mode

     *   DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020+DXGI_FORMAT_R10G10B10A2_UNORM
     *   This would be very efficient for the application you discuss

  *   Windows has no "rec2100-hlg" support

     *   That is, no support for RGB HLG buffers (there does exist YUV HLG support, of course)

  *   macOS had a "rec2100-pq" CGColorSpace (kCGColorSpaceITUR_2020_PQ_EOTF) but that has been deprecated

     *   PQ can, of course, be used with video<http://kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ>

  *   macOS also deprecated "rec2100-hlg" CGColorSpace (kCGColorSpaceITUR_2020_HLG) but has a "display-p3-hlg" space.
All of this is to say that the browser would, in the background, be converting into a floating point format behind the user's back most of the time. But it may be that TV-embedded browsers will have a particular signal and pixel format that is desired.

So I'm in favor of this being exposed if it's well-supported.