Re: High Dynamic Range imaging and ICC colour management

Peak white; Diffuse white is chosen globally at app startup based on a
handful of factors, such as output mode (HDR10, DVLL, etc).


On Tue, Mar 5, 2019, 6:43 AM Craig Revie <Craig.Revie@ffei.co.uk> wrote:

> Hi Joe,
>
>
>
> Thanks for your reply and a good overview of the work being done at
> Netflix. It is not my intent to offer any solutions at this stage in the
> discussion, but it seems that you are in a similar place to others working
> in this area, some of whom are members of this group, in that you have
> adopted ICC and (for perfectly good reason) given it a proprietary flavour.
> It seems to me that there may be scope for a suitable group of stakeholders
> in this area to work with the ICC to develop an open framework providing
> interoperability.
>
>
>
> As well as Netflix, I am aware of work in this area by a number of other
> companies and it would seem that working together as a group may produce a
> much better solution for all concerned.
>
>
>
> One question about your use of the luminance tag. Are you using this for
> peak white or diffuse white? Either way, it seems that at least for PQ
> image encoding both values are needed.
>
>
>
> Best regards,
>
> _Craig
>
>
>
> *From:* Joe Drago <jdrago@netflix.com>
> *Sent:* 05 March 2019 00:38
> *To:* Mark Watson <watsonm@netflix.com>
> *Cc:* Craig Revie <Craig.Revie@ffei.co.uk>; public-colorweb@w3.org
> *Subject:* Re: High Dynamic Range imaging and ICC colour management
>
>
>
> Hello Craig (and all other colorweb folks) --
>
>
>
> In the meantime, please let me know if this is something you would like to
> work on or if you are already aware of other solutions or proposed
> solutions to this problem.
>
>
>
> I figured I'd chime in here as I was responsible for the implementation of
> HDR image packaging here at Netflix, and my approach leverages (if not
> abuses!) ICC profiles. Here are some jump-off points which I'll reference
> in my explanation:
>
>
>
>
> https://medium.com/netflix-techblog/enhancing-the-netflix-ui-experience-with-hdr-1e7506ad3e8
>
> https://joedrago.github.io/colorist/
>
>
>
> Just as a clarification for these links: The first link is our Netflix
> Blog post where we discuss the strategy we employed, and the second link is
> the open source tool (colorist) I created to implement that strategy.
> Seeing as you're all experts in ICC and I'm squarely in the newcomer group,
> I'll spare any explanations on ICC's guts and skip to my approach to the
> situation and various learnings/conclusions I came to while tackling this
> problem. I'll recap my (ab)use of the ICC profile from the colorist site's
> Overview block in this email below.
>
>
>
> My mandate here at Netflix was to come up with a way to pack an HDR image
> coming from a source video that was HDR10 (BT.2020 PQ, 10000 nits
> display-referred) into any file format I deemed capable, and then see where
> we could leverage preexisting standards as much as possible. Seeing as both
> our source data and our output sink both operated in absolute luminance
> (display-referred with a range of [0-10000] nits), I based my solution
> around that.
>
>
>
> I learned early on that ICC profiles could perfectly describe our color
> primaries, and after being disappointed that there wasn't a
> parametricCurveType for PQ, I realized if I was willing to use more bits
> per channel (16 instead of 10), that such a severe curve as PQ wasn't
> necessary for the transfer function, and I could instead stick with a more
> friendly curve that maintains enduser blending expectations (BT.1886 or
> even just a plain 2.2 gamma). This solved the insufficient precision in the
> curv LUTs cited in your slide deck, as (for example) a Type1 'para' with a
> value of 2.2 is quite friendly and packs nice and tiny for image file
> embedding. However, to fully realize a display-referred image with a
> standard ICC profile, I needed one more piece of information: the absolute
> luminance range.
>
>
>
> I noticed that ICC profiles have had a 'lumi' tag, but despite it being
> populated in some very common ICC profile chunks out there (including on
> color.org), it was merely informative and modern image editing software /
> CMMs appeared to ignore the value completely. However, if I were to honor &
> leverage this value during image creation and conversion, I could maintain
> a proper display-referred image file, which I could then convert on the fly
> when rendering on game consoles, and thus maintain creative intent exactly.
>
>
>
> In summary: I'm storing the "correct" value in the lumi tag based on the
> spec, but I'm actually *using* it to derive a luminance scale when
> converting and rendering, thus making the ICC profile display-referred.
>
>
>
> To be clear, I'm not trying to make the case that this is the one-true-way
> forward for HDR or anything; I recognize the conveniences that HLG provides
> and that using absolute luminance signs up various decoders for luminance
> scaling and tonemapping to interpret the image in the first place. That
> said, I do think that absolute luminance in image files is the most
> accurate way to maintain creative intent, and for any BMFF-derived image
> file (HEIF, MIAF, AVIF) using an 'nclx' type colr box, they already offer a
> means to specify a PQ transfer_characteristics value which implies absolute
> luminance anyway, so this is something we need to honor in the future
> regardless of my ICC lumi tag abuse.
>
>
>
> As a wish list for ICC profiles in this very specific, display-referred
> image file world, here's some things I wish I had:
>
>
>
> * I wish there existed a tag (such as the lumi tag) which indicated that
> the associated pixel data was display-referred, and provided a real
> luminance range that must be honored. While I'm using the lumi tag
> currently, there are 30 years of files with embedded ICC profiles that may
> or may not contain "interesting" lumi tag values, so ideally it'd be a new
> tag type, or perhaps co-opting a lesser-used tag (viewingConditionsTag?).
>
>
>
> * A new official parametricCurveType type value for PQ, so CMMs could do
> proper precision PQ math without making ICC profile chunks really large.
>
>
>
> My disclaimer for the second wish there is that I only think PQ is
> necessary for HDR when at 10 and 12 bpc. Once you have 16bpc, I think you
> can get away with murder and just use a simple 2.2 gamma; everything looks
> fantastic and trivially blends in the 'incorrect' way people are used to.
> Our final deliverables in production for our HDR images ended up being .JP2
> files encoded with 16bpc BT2020 2.2 gamma, display-referred at [0-10000]
> nits. I still think a PQ para type should exist.
>
>
>
> Anyway, this is where I landed on packaging HDR images. You can see my
> full implementation for conversion and reporting in the colorist repo
> linked above, including my (new and incomplete) method for detecting
> special ICC profile combinations that can be repackaged as compliant and
> ICC-free BT.2100 nclx colr boxes in AVIF files. I'm confident that I'm
> maintaining the fidelity of these stored images and rendering the
> correctly, and I'm also confident that I'm ever-so-slightly abusing ICC
> profiles to achieve it.
>
>
>
>
>
>
>
>
>
> On Mon, Mar 4, 2019 at 2:04 PM Mark Watson <watsonm@netflix.com> wrote:
>
> Hi Craig,
>
>
>
> This is very interesting and something we have been looking at in the
> context of applications running on Games Consoles and Set Top Boxes where
> the output is locked to HDR (when the TV supports), to avoid the problems
> associated with HDR / SDR switching on the HDMI link.
>
>
>
> I'd like to introduce my colleague Joe Drago (who joined this list after
> your post, which is the main reason for this message). He has some comments
> / questions.
>
>
>
> ...Mark
>
>
>
> On Mon, Mar 4, 2019 at 6:07 AM Craig Revie <Craig.Revie@ffei.co.uk> wrote:
>
> I have been asked to post a presentation that I made in last month’s ICC
> meeting to this group. The links below are to my presentation and a
> recording of the meeting including discussion.
>
>
> http://www.color.org/groups/displays/20190215_High_Dynamic_Range_imaging_and_Hybrid_Log-Gamma.pdf
>
> http://www.color.org/groups/displays/High_Dynamic_Range_Imaging_and_ICC.mp4
>
>
>
> This presentation was the result of discussion with several experts from
> different areas, some of whom are members of this group. The problem that
> we are trying to address is that of effective presentation of High Dynamic
> Range (HDR) and Standard Dynamic Range (SDR) images or video sequences on
> the same display. This is particularly relevant where the content creator
> is not able to control the way in which the content is rendered to the
> display or to broadcast content appropriate to a specific display.
>
>
>
> An effective solution to this problem requires an architecture similar to
> that of the ICC where an input transform maps colour content to a
> well-defined ‘connection space’ and an output transform maps from this
> connection space to a device (usually a display or printer). Although the
> presentation is from an ICC perspective, other models could be used.
>
>
>
> This is a very basic outline of one possible solution to this problem and
> at this stage I am interested to identify others (from this group and
> elsewhere) who would be interested in working further on this topic. The
> response from the ICC was very positive and I believe that the ICC would be
> willing to host a forum for this discussion, especially to explore whether
> the current ICC model (with minor extensions) would be useful.
>
>
>
> I would be happy to present these ideas in more detail if anyone is
> interested.
>
>
>
> In the meantime, please let me know if this is something you would like to
> work on or if you are already aware of other solutions or proposed
> solutions to this problem.
>
>
>
> Best regards,
>
> Craig Revie
> ------------------------------
>
> [image: FFEI Limited] <http://www.ffei.co.uk>
>
> *CONFIDENTIALITY AND DISCLAIMER NOTICE*
>
> This message and any attachment is confidential and is protected by
> copyright. If you are not the intended recipient, please email the sender
> and delete this message and any attachment from your system.
>
> Dissemination and or copying of this email is prohibited if you are not
> the intended recipient. We believe, but do not warrant, that this email and
> any attachments are virus free. You should take full responsibility for
> virus checking.
>
> No responsibility is accepted by FFEI Ltd for personal emails or emails
> unconnected with FFEI Limited's business.
>
> FFEI Limited is a limited company registered in England and Wales
> (Registered Number: 3244452).
>
> [image: Join us on Linked In] <http://www.linkedin.com/company/ffei>[image:
> Follow @FFEI_ltd] <https://twitter.com/FFEI_ltd>[image: FFEI YouTube
> Channel] <http://www.youtube.com/user/FFEIPrintTechnology>
>
> *Registered Office: The Cube, Maylands Avenue, Hemel Hempstead,
> Hertfordshire, HP2 7DF, England. *
>
>