Re: Linear canvas vs non-linear pixels.

On Thu, Mar 18, 2021 at 10:31 AM Lars Borg <borg@adobe.com> wrote:

> FYI 10-bit YUV 4:2:0 is not 24bpp. V210 is a common packing for 10-bit YUV
> and it wastes 2 bits/32 bits.
>
>
>
> In general, bit rate for 4:2:0 is 1/2 of 4:4:4, bit rate for 4:2:2 is 2/3
> of 4:4:4, ignoring packing bits.
>

This was with respect to GPU memory usage. On Windows
<https://docs.microsoft.com/en-us/windows/win32/medfound/10-bit-and-16-bit-yuv-video-formats>,
10-bit YUV has 16 bits for each Y, U, and V sample (so 16 for Y, and
16/(2*2) for U and 16/(2*2) for V, for 24 total). Same deal on macOS
<https://source.chromium.org/chromium/chromium/src/+/master:ui/gfx/mac/io_surface.cc;l=59;drc=3ef5db76bb776fc42804f73d168fef5829816145>
.

GPUs are particular about their packing formats (e.g: support for
WebGL's alpha:false mode is a headache, because 24-bit RGB textures haven't
existed in hardware for years. It's all RGBX, and sometimes that X isn't as
ignored as it should be).


>
>
> Lars
>
>
>
>
>
> *From: *Christopher Cameron <ccameron@google.com>
> *Date: *Wednesday, March 17, 2021 at 7:13 PM
> *To: *Leonard Rosenthol <lrosenth@adobe.com>
> *Cc: *Pierre Lemieux <pal@sandflow.com>, "public-colorweb@w3.org" <
> public-colorweb@w3.org>
> *Subject: *Re: Linear canvas vs non-linear pixels.
> *Resent-From: *<public-colorweb@w3.org>
> *Resent-Date: *Wednesday, March 17, 2021 at 7:12 PM
>
>
>
> Yes, macOS and iOS are roughly identical. Android supports float16 and
> unorm10 (see here
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fview%2FSurfaceControl.Builder%23setFormat(int)&data=04%7C01%7Cborg%40adobe.com%7Cc04e04acfa814c64f4f608d8e9cc8453%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637516411897668143%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XURdxASSnhgZ7qYI9H57gxS0RxuU3SsfDT8FeDrQ25s%3D&reserved=0>
> for SurfaceControl API, which I think is the modern relevant one).
>
>
>
> A device without a GPU won't be doing HDR canvas. I looked around for
> low-end phones without a GPU, and the last ones I could find were from 2009.
>
>
>
> To the point you're making: Is having a GPU capable of float16 an onerous
> requirement? I don't think so. HDR cannot be done in 8 bits. GPUs that
> don't support float16 also don't support >8 bit colors (OpenGL ES 2.0
> requires neither, OpenGL ES 3.0 requires both).
>
>
>
> You're not wrong though -- float16 should scare you. The principal reason
> to avoid float16 isn't "the GPU can't handle it as a texture format" but
> rather "the memory bandwidth requirement is going to kill the system."
> There is suffering just from the fact that canvas is perforce RGB. An 8-bit
> YUV 4:2:0 frame is 12bits/pixel, an RGBA8 canvas is 32 bits/pixel. A 10-bit
> YUV 4:2:0 frame is 24bits/pixel (IIUC there's some dead space there --
> haven't touched it with my hands, so I may be wrong) while an RGB10A2
> canvas is is 32bits/pixel and an RGBA16F canvas is 64bits/pixel. That's
> where the pain will be.
>
>
>
> I don't think we're going to be making float16 a requirement. A
> 10-bit-per-channel PQ or HLG format is quite reasonable, and perhaps there
> will be a device that will refuse to create anything but those formats
> (e.g: any device that has "just enough" memory bandwidth).
>
>
>
>
>
>
>
> On Wed, Mar 17, 2021 at 1:48 AM Leonard Rosenthol <lrosenth@adobe.com>
> wrote:
>
> Chris – can you also comment (if you know) about mobile OS support?  For
> example, does the MacOS state also match iOS?   What about Android?   What
> about performance concerns of floating point operations and lack of GPU on
> many lower end Android devices?
>
>
>
> Leonard
>
>
>
> *From: *Christopher Cameron <ccameron@google.com>
> *Date: *Wednesday, March 17, 2021 at 2:40 AM
> *To: *Pierre-Anthony Lemieux <pal@sandflow.com>
> *Cc: *"public-colorweb@w3.org" <public-colorweb@w3.org>
> *Subject: *Re: Linear canvas vs non-linear pixels.
> *Resent-From: *<public-colorweb@w3.org>
> *Resent-Date: *Wednesday, March 17, 2021 at 2:39 AM
>
>
>
>
>
>
>
> On Tue, Mar 16, 2021 at 1:50 PM Pierre-Anthony Lemieux <pal@sandflow.com>
> wrote:
>
> Thanks to Chris for drafting a concrete proposal [0]. Below is a
> potential challenge when handling PQ HDR content.
>
> Starting from mastering, video pixels are coded as non-linear integer
> code values, e.g. HLG, PQ, etc..
>
> These pixels ideally remain unmodified until they are rendered to the
> display since each conversion carries a risk that fidelity will be
> lost. This is the case even in the case where the color space and the
> pixel encoded remain the same.
>
> The strawman currently requires that PQ pixels be converted to linear
> values when written to Canvas:
>
>    PQ CV ----F----> linear value -----G-----> PQ CV
>
> This is non-trivial and computationally expensive since F is highly
> non-linear and covers many orders of magnitude. It also requires G to
> be the exact inverse of F, which is not always simple if F and G are
> provided by different implementations.
>
> How do we provide a means of avoiding this complexity and potential
> loss of fidelity?
>
>
>
> Could one approach be to add optional support for non-linear pixel
> encodings, e.g. add 'rec2100--pq' to [1]. In other words,
> implementation would need to support 'rec2020-linear' but could also
> support 'rec2100--pq'.
>
>
>
> Adding a 'rec2100-pq' and 'rec2100-hlg' color is quite reasonable.
>
>
>
> Support is spotty on desktop operating systems.
>
> · Windows has a "rec2100-pq + rgb10-alpha2" mode
>
> o DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020+DXGI_FORMAT_R10G10B10A2_UNORM
>
> o This would be very efficient for the application you discuss
>
> · Windows has no "rec2100-hlg" support
>
> o That is, no support for RGB HLG buffers (there does exist YUV HLG
> support, of course)
>
> · macOS had a "rec2100-pq" CGColorSpace (kCGColorSpaceITUR_2020_PQ_EOTF)
> but that has been deprecated
>
> o PQ can, of course, be used with video
> <http://kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ>
>
> · macOS also deprecated "rec2100-hlg" CGColorSpace
> (kCGColorSpaceITUR_2020_HLG) but has a "display-p3-hlg" space.
>
> All of this is to say that the browser would, in the background, be
> converting into a floating point format behind the user's back most of the
> time. But it may be that TV-embedded browsers will have a particular signal
> and pixel format that is desired.
>
>
>
> So I'm in favor of this being exposed if it's well-supported.
>
>
>
>
>
>