Re: Linear canvas vs non-linear pixels.

Yes, macOS and iOS are roughly identical. Android supports float16 and
unorm10 (see here
<https://developer.android.com/reference/android/view/SurfaceControl.Builder#setFormat(int)>
for SurfaceControl API, which I think is the modern relevant one).

A device without a GPU won't be doing HDR canvas. I looked around for
low-end phones without a GPU, and the last ones I could find were from 2009.

To the point you're making: Is having a GPU capable of float16 an onerous
requirement? I don't think so. HDR cannot be done in 8 bits. GPUs that
don't support float16 also don't support >8 bit colors (OpenGL ES 2.0
requires neither, OpenGL ES 3.0 requires both).

You're not wrong though -- float16 should scare you. The principal reason
to avoid float16 isn't "the GPU can't handle it as a texture format" but
rather "the memory bandwidth requirement is going to kill the system."
There is suffering just from the fact that canvas is perforce RGB. An 8-bit
YUV 4:2:0 frame is 12bits/pixel, an RGBA8 canvas is 32 bits/pixel. A 10-bit
YUV 4:2:0 frame is 24bits/pixel (IIUC there's some dead space there --
haven't touched it with my hands, so I may be wrong) while an RGB10A2
canvas is is 32bits/pixel and an RGBA16F canvas is 64bits/pixel. That's
where the pain will be.

I don't think we're going to be making float16 a requirement. A
10-bit-per-channel PQ or HLG format is quite reasonable, and perhaps there
will be a device that will refuse to create anything but those formats
(e.g: any device that has "just enough" memory bandwidth).



On Wed, Mar 17, 2021 at 1:48 AM Leonard Rosenthol <lrosenth@adobe.com>
wrote:

> Chris – can you also comment (if you know) about mobile OS support?  For
> example, does the MacOS state also match iOS?   What about Android?   What
> about performance concerns of floating point operations and lack of GPU on
> many lower end Android devices?
>
>
>
> Leonard
>
>
>
> *From: *Christopher Cameron <ccameron@google.com>
> *Date: *Wednesday, March 17, 2021 at 2:40 AM
> *To: *Pierre-Anthony Lemieux <pal@sandflow.com>
> *Cc: *"public-colorweb@w3.org" <public-colorweb@w3.org>
> *Subject: *Re: Linear canvas vs non-linear pixels.
> *Resent-From: *<public-colorweb@w3.org>
> *Resent-Date: *Wednesday, March 17, 2021 at 2:39 AM
>
>
>
>
>
>
>
> On Tue, Mar 16, 2021 at 1:50 PM Pierre-Anthony Lemieux <pal@sandflow.com>
> wrote:
>
> Thanks to Chris for drafting a concrete proposal [0]. Below is a
> potential challenge when handling PQ HDR content.
>
> Starting from mastering, video pixels are coded as non-linear integer
> code values, e.g. HLG, PQ, etc..
>
> These pixels ideally remain unmodified until they are rendered to the
> display since each conversion carries a risk that fidelity will be
> lost. This is the case even in the case where the color space and the
> pixel encoded remain the same.
>
> The strawman currently requires that PQ pixels be converted to linear
> values when written to Canvas:
>
>    PQ CV ----F----> linear value -----G-----> PQ CV
>
> This is non-trivial and computationally expensive since F is highly
> non-linear and covers many orders of magnitude. It also requires G to
> be the exact inverse of F, which is not always simple if F and G are
> provided by different implementations.
>
> How do we provide a means of avoiding this complexity and potential
> loss of fidelity?
>
>
>
> Could one approach be to add optional support for non-linear pixel
> encodings, e.g. add 'rec2100--pq' to [1]. In other words,
> implementation would need to support 'rec2020-linear' but could also
> support 'rec2100--pq'.
>
>
>
> Adding a 'rec2100-pq' and 'rec2100-hlg' color is quite reasonable.
>
>
>
> Support is spotty on desktop operating systems.
>
>    - Windows has a "rec2100-pq + rgb10-alpha2" mode
>
>
>    -
>       DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020+DXGI_FORMAT_R10G10B10A2_UNORM
>       - This would be very efficient for the application you discuss
>
>
>    - Windows has no "rec2100-hlg" support
>
>
>    - That is, no support for RGB HLG buffers (there does exist YUV HLG
>       support, of course)
>
>
>    - macOS had a "rec2100-pq" CGColorSpace
>    (kCGColorSpaceITUR_2020_PQ_EOTF) but that has been deprecated
>
>
>    - PQ can, of course, be used with video
>       <http://kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ>
>
>
>    - macOS also deprecated "rec2100-hlg" CGColorSpace
>    (kCGColorSpaceITUR_2020_HLG) but has a "display-p3-hlg" space.
>
> All of this is to say that the browser would, in the background, be
> converting into a floating point format behind the user's back most of the
> time. But it may be that TV-embedded browsers will have a particular signal
> and pixel format that is desired.
>
>
>
> So I'm in favor of this being exposed if it's well-supported.
>
>
>
>
>