Re: Linear canvas vs non-linear pixels.

On Tue, Mar 16, 2021 at 1:50 PM Pierre-Anthony Lemieux <pal@sandflow.com>
wrote:

> Thanks to Chris for drafting a concrete proposal [0]. Below is a
> potential challenge when handling PQ HDR content.
>
> Starting from mastering, video pixels are coded as non-linear integer
> code values, e.g. HLG, PQ, etc..
>
> These pixels ideally remain unmodified until they are rendered to the
> display since each conversion carries a risk that fidelity will be
> lost. This is the case even in the case where the color space and the
> pixel encoded remain the same.
>
> The strawman currently requires that PQ pixels be converted to linear
> values when written to Canvas:
>
>    PQ CV ----F----> linear value -----G-----> PQ CV
>
> This is non-trivial and computationally expensive since F is highly
> non-linear and covers many orders of magnitude. It also requires G to
> be the exact inverse of F, which is not always simple if F and G are
> provided by different implementations.
>
> How do we provide a means of avoiding this complexity and potential
> loss of fidelity?
>


> Could one approach be to add optional support for non-linear pixel
> encodings, e.g. add 'rec2100--pq' to [1]. In other words,
> implementation would need to support 'rec2020-linear' but could also
> support 'rec2100--pq'.


Adding a 'rec2100-pq' and 'rec2100-hlg' color is quite reasonable.

Support is spotty on desktop operating systems.

   - Windows has a "rec2100-pq + rgb10-alpha2" mode
   -
      DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020+DXGI_FORMAT_R10G10B10A2_UNORM
      - This would be very efficient for the application you discuss
      - Windows has no "rec2100-hlg" support
   - That is, no support for RGB HLG buffers (there does exist YUV HLG
      support, of course)
      - macOS had a "rec2100-pq" CGColorSpace
   (kCGColorSpaceITUR_2020_PQ_EOTF) but that has been deprecated
   - PQ can, of course, be used with video
      <http://kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ>
      - macOS also deprecated "rec2100-hlg" CGColorSpace
   (kCGColorSpaceITUR_2020_HLG) but has a "display-p3-hlg" space.

All of this is to say that the browser would, in the background, be
converting into a floating point format behind the user's back most of the
time. But it may be that TV-embedded browsers will have a particular signal
and pixel format that is desired.

So I'm in favor of this being exposed if it's well-supported.