Pre-meeting Thoughts on HDR Canvas

Hi all --

To avoid me rambling for a few minutes at some point in the meeting
(although it might happen anyway), I figured if I listed a high level
outline of how I perceive the HDR Canvas issue, it'd help frame the
upcoming conversation. It also happens to coincide nicely with the
comprehensive slides Chris Cameron is planning to share. This is mostly
top-of-head, so I reserve the right to add things to this list in the
future. [grin]

I think when talking about color profiles for canvas, there are (at least)
three distinct *use cases* that each have their own needs:

* The canvas' working profile / pixel format
  * Determines luminance gamut / luminance range
  * Determines blending
  * Determines precision (banding)

* User-defined colors (CSS, constants in JS, etc)
  * Should be really easy to reason about

* Export / Save
  * Pixel sampling from a canvas?
  * Save to disk while retaining HDR (or not)

Before I dig into any of these, I wanted to note that I think treating
these use cases as separable issues will help a lot. The goal here isn't to
come up with a single profile or solution that is a compromise for all of
these, but instead to choose the best answers for each use case and have
them interoperate behind the scenes. Also note that plain ol' srgb will
always be a choice for all of these use cases (of course).

If we separate the issues, the canvas' working profile becomes quite easy.
As long as we allow for overranging (below 0 and above 1) *or *use a
sufficiently large gamut + luminance range, the only real interesting
questions here are precision and transfer function. If we go the
overranging route (very likely), I believe fp16 is sufficient precision for
even a linear TF, so we're all set on precision here as well. As for
transfer functions on the canvas itself, I don't know if we need to offer
anything more than srgb and linear, honestly. The srgb TF provides endusers
with the exact kind of "incorrect" blending they've been used to, and
linear allows for proper light math and 3D fun.

User-defined colors are a bit trickier. I recommend that the choices here
are a bit more varied, as people will be expected to type them into CSS,
JS, and inspect/reason about them. This is something I've had to deal with
internally at my work, and I think getting this one right will go a long
way to adoption. People are *so *used to srgb that training them to use
wider gamuts and larger ranges is going to be really hard if we're
expecting them to supply impossible-to-grok negative values in channels,
for example. If we can't reason about them *ourselves*, we can't expect
them to.

My suggestion here would be to lean into extended linear profiles here, and
perhaps offer more selections than we need to offer on the canvas. I
believe Chris' slides cite srgb-linear, display-p3, and bt2020-linear as
choices (which are great), but I think we might also want display-p3-linear.
I think people just getting started on WCG/HDR theory can grok (display-p3
1 0 0), and I think if we standardized the luminance of (1.0,1.0,1.0) (as
Chris' slides will discuss), people will *also *be able to reason pretty
easily around something like (p3-linear 5 5 5). We're going to be asking a
lot of people that have only ever thought in srgb up until now, but I'm
sure once these standards are in place that we'll have some great crash
courses out there.

Export / Save is a weird one, but important. Chris' slides will go into
detail on this one (ImageData / ImageBitmap), but I just wanted to note
that transfer functions play a much different role here, and due to file
format limitations, we might want a few more selections here, notably PQ
and HLG. I obviously have a bit of a bias with HEIF-derived formats (AVIF ,
HEIC), but I think 10bpc P3/BT2020 PQ/HLG images are going to be too
useful/common as an export format to not have some bidirectional mechanism
for those profiles, even if we don't actually do the direct final encoding
to any of those formats. Being able to ask the canvas for "please give me
unorm16 BT2020 PQ" would be an incredible feature in today's world of
WASM-compiled encoders.

If you've made it this far, thanks for reading! I look forward to Chris'
presentation and following conversation.