Re: High Dynamic Range imaging and ICC colour management

Hello Craig (and all other colorweb folks) --

In the meantime, please let me know if this is something you would like to
> work on or if you are already aware of other solutions or proposed
> solutions to this problem.
>

I figured I'd chime in here as I was responsible for the implementation of
HDR image packaging here at Netflix, and my approach leverages (if not
abuses!) ICC profiles. Here are some jump-off points which I'll reference
in my explanation:

https://medium.com/netflix-techblog/enhancing-the-netflix-ui-experience-with-hdr-1e7506ad3e8
https://joedrago.github.io/colorist/

Just as a clarification for these links: The first link is our Netflix Blog
post where we discuss the strategy we employed, and the second link is the
open source tool (colorist) I created to implement that strategy. Seeing as
you're all experts in ICC and I'm squarely in the newcomer group, I'll
spare any explanations on ICC's guts and skip to my approach to the
situation and various learnings/conclusions I came to while tackling this
problem. I'll recap my (ab)use of the ICC profile from the colorist site's
Overview block in this email below.

My mandate here at Netflix was to come up with a way to pack an HDR image
coming from a source video that was HDR10 (BT.2020 PQ, 10000 nits
display-referred) into any file format I deemed capable, and then see where
we could leverage preexisting standards as much as possible. Seeing as both
our source data and our output sink both operated in absolute luminance
(display-referred with a range of [0-10000] nits), I based my solution
around that.

I learned early on that ICC profiles could perfectly describe our color
primaries, and after being disappointed that there wasn't a
parametricCurveType for PQ, I realized if I was willing to use more bits
per channel (16 instead of 10), that such a severe curve as PQ wasn't
necessary for the transfer function, and I could instead stick with a more
friendly curve that maintains enduser blending expectations (BT.1886 or
even just a plain 2.2 gamma). This solved the insufficient precision in the
curv LUTs cited in your slide deck, as (for example) a Type1 'para' with a
value of 2.2 is quite friendly and packs nice and tiny for image file
embedding. However, to fully realize a display-referred image with a
standard ICC profile, I needed one more piece of information: the absolute
luminance range.

I noticed that ICC profiles have had a 'lumi' tag, but despite it being
populated in some very common ICC profile chunks out there (including on
color.org), it was merely informative and modern image editing software /
CMMs appeared to ignore the value completely. However, if I were to honor &
leverage this value during image creation and conversion, I could maintain
a proper display-referred image file, which I could then convert on the fly
when rendering on game consoles, and thus maintain creative intent exactly.

In summary: I'm storing the "correct" value in the lumi tag based on the
spec, but I'm actually *using* it to derive a luminance scale when
converting and rendering, thus making the ICC profile display-referred.

To be clear, I'm not trying to make the case that this is the one-true-way
forward for HDR or anything; I recognize the conveniences that HLG provides
and that using absolute luminance signs up various decoders for luminance
scaling and tonemapping to interpret the image in the first place. That
said, I do think that absolute luminance in image files is the most
accurate way to maintain creative intent, and for any BMFF-derived image
file (HEIF, MIAF, AVIF) using an 'nclx' type colr box, they already offer a
means to specify a PQ transfer_characteristics value which implies absolute
luminance anyway, so this is something we need to honor in the future
regardless of my ICC lumi tag abuse.

As a wish list for ICC profiles in this very specific, display-referred
image file world, here's some things I wish I had:

* I wish there existed a tag (such as the lumi tag) which indicated that
the associated pixel data was display-referred, and provided a real
luminance range that must be honored. While I'm using the lumi tag
currently, there are 30 years of files with embedded ICC profiles that may
or may not contain "interesting" lumi tag values, so ideally it'd be a new
tag type, or perhaps co-opting a lesser-used tag (viewingConditionsTag?).

* A new official parametricCurveType type value for PQ, so CMMs could do
proper precision PQ math without making ICC profile chunks really large.

My disclaimer for the second wish there is that I only think PQ is
necessary for HDR when at 10 and 12 bpc. Once you have 16bpc, I think you
can get away with murder and just use a simple 2.2 gamma; everything looks
fantastic and trivially blends in the 'incorrect' way people are used to.
Our final deliverables in production for our HDR images ended up being .JP2
files encoded with 16bpc BT2020 2.2 gamma, display-referred at [0-10000]
nits. I still think a PQ para type should exist.

Anyway, this is where I landed on packaging HDR images. You can see my full
implementation for conversion and reporting in the colorist repo linked
above, including my (new and incomplete) method for detecting special ICC
profile combinations that can be repackaged as compliant and ICC-free
BT.2100 nclx colr boxes in AVIF files. I'm confident that I'm maintaining
the fidelity of these stored images and rendering the correctly, and I'm
also confident that I'm ever-so-slightly abusing ICC profiles to achieve
it.




On Mon, Mar 4, 2019 at 2:04 PM Mark Watson <watsonm@netflix.com> wrote:

> Hi Craig,
>
> This is very interesting and something we have been looking at in the
> context of applications running on Games Consoles and Set Top Boxes where
> the output is locked to HDR (when the TV supports), to avoid the problems
> associated with HDR / SDR switching on the HDMI link.
>
> I'd like to introduce my colleague Joe Drago (who joined this list after
> your post, which is the main reason for this message). He has some comments
> / questions.
>
> ...Mark
>
> On Mon, Mar 4, 2019 at 6:07 AM Craig Revie <Craig.Revie@ffei.co.uk> wrote:
>
>> I have been asked to post a presentation that I made in last month’s ICC
>> meeting to this group. The links below are to my presentation and a
>> recording of the meeting including discussion.
>>
>>
>> http://www.color.org/groups/displays/20190215_High_Dynamic_Range_imaging_and_Hybrid_Log-Gamma.pdf
>>
>>
>> http://www.color.org/groups/displays/High_Dynamic_Range_Imaging_and_ICC.mp4
>>
>>
>>
>> This presentation was the result of discussion with several experts from
>> different areas, some of whom are members of this group. The problem that
>> we are trying to address is that of effective presentation of High Dynamic
>> Range (HDR) and Standard Dynamic Range (SDR) images or video sequences on
>> the same display. This is particularly relevant where the content creator
>> is not able to control the way in which the content is rendered to the
>> display or to broadcast content appropriate to a specific display.
>>
>>
>>
>> An effective solution to this problem requires an architecture similar to
>> that of the ICC where an input transform maps colour content to a
>> well-defined ‘connection space’ and an output transform maps from this
>> connection space to a device (usually a display or printer). Although the
>> presentation is from an ICC perspective, other models could be used.
>>
>>
>>
>> This is a very basic outline of one possible solution to this problem and
>> at this stage I am interested to identify others (from this group and
>> elsewhere) who would be interested in working further on this topic. The
>> response from the ICC was very positive and I believe that the ICC would be
>> willing to host a forum for this discussion, especially to explore whether
>> the current ICC model (with minor extensions) would be useful.
>>
>>
>>
>> I would be happy to present these ideas in more detail if anyone is
>> interested.
>>
>>
>>
>> In the meantime, please let me know if this is something you would like
>> to work on or if you are already aware of other solutions or proposed
>> solutions to this problem.
>>
>>
>>
>> Best regards,
>>
>> Craig Revie
>> ------------------------------
>> [image: FFEI Limited] <http://www.ffei.co.uk>
>> *CONFIDENTIALITY AND DISCLAIMER NOTICE*
>>
>> This message and any attachment is confidential and is protected by
>> copyright. If you are not the intended recipient, please email the sender
>> and delete this message and any attachment from your system.
>>
>> Dissemination and or copying of this email is prohibited if you are not
>> the intended recipient. We believe, but do not warrant, that this email and
>> any attachments are virus free. You should take full responsibility for
>> virus checking.
>>
>> No responsibility is accepted by FFEI Ltd for personal emails or emails
>> unconnected with FFEI Limited's business.
>>
>> FFEI Limited is a limited company registered in England and Wales
>> (Registered Number: 3244452).
>> [image: Join us on Linked In] <http://www.linkedin.com/company/ffei> [image:
>> Follow @FFEI_ltd] <https://twitter.com/FFEI_ltd> [image: FFEI YouTube
>> Channel] <http://www.youtube.com/user/FFEIPrintTechnology>
>> Registered Office: The Cube, Maylands Avenue, Hemel Hempstead,
>> Hertfordshire, HP2 7DF, England.
>>
>