Re: Mapping of SDR into HDR

While I agree with pretty much everything said here, it doesn't seem to
offer a lot of solutions to the problems at hand.

It seems to me that the "neutral un-rendered form" you speak of is very
similar to the composited form suggested in my "principles of hdr in
chrome" document.
My short an inaccurate "SDR should be mapped into HDR as when played in a
reference environment" might be misleading in this regard, because what I
suggest in my document is actually that hey be composited
into scRGB or extended sRGB, which would seem pretty similar to your
neutral un-rendered form.

What I'm not sure of is weather there is a functional difference between
what you suggest and what I suggest (however clumsily) in the my document,
or if the difference is simply philosophical.

       /Hubbe

On Thu, Nov 23, 2017 at 7:51 AM, Tim Borer <tim.borer@bbc.co.uk> wrote:

> Dear All,
>
>
> I don’t agree that “SDR should be mapped into HDR as when played in a
> reference environment, the composited result should then be adapted to the
> viewing environment”. In particular I disagree that, in compositing, there
> needs to be a reference environment.
>
>
> If you mapped to HDR in a reference environment you would have to invert
> that rendering, and then re-render for the desired display and viewing
> environment. Instead I think that all signals (SDR and HDR) should be
> converted to a neural un-rendered form. Then the display can render, as
> appropriate, to that display and viewing environment.
>
>
> An appropriate un-rendered form is a linear scene referred signal. A scene
> referred signal is what is, almost universally, captured by cameras (both
> still and video). It is not the signal that is displayed on a reference
> monitor in reference condition. All signals (SDR and HDR) may be converted
> to a linear scene referred form. Scene referred linear light is what is
> generated in CGI, where it is commonly stored as half floats with nominal
> diffuse white at a value of 1.0. A format for linear scene referred video
> is specified in BT.2100.
>
>
> I beg to differ with Craig Todd’s interpretation of SDR and of the ITU
> documents to which he refers. There is, to my mind, a fundamental
> difference, and a fundamental difference in philosophy, between PQ and
> HLG/SDR. Claims that display referred PQ is the same as scene referred SDR
> seem to have engendered a great deal of confusion over the past few years.
>
>
> Fundamentally PQ is display referred, whereas SDR and HLG are scene
> referred. This means that the PQ signal is defined in terms of the signal
> that is displayed on a reference monitor in reference viewing conditions,
> whereas a scene referred signal, such as SDR or HLG, is that which is
> captured by the camera. Since it is necessary to render a signal for the
> actual viewing conditions and viewing environment, and this rendering is
> non-linear, scene and display referred signals are not linearly related to
> each other. They are, fundamentally, different.
>
>
> A good quote, which I think clarifies this issue, comes from “Cinematic
> Color” (full reference and URL below). It says: “Broadly speaking, film
> negatives encode an HDR scene-referred image, and the print embodies a
> display-referred tone mapping.” Note that the negative, corresponding to
> HLG/SDR, is scene referred and that the print, corresponding to the PQ
> signal, is display referred. The same paper provides a good discussion of
> scene versus display referred signals and also says: “This mismatch between
> the dynamic range of the real world and the dynamic range of
> display-technology makes working in display-referred color spaces (even
> linear ones) ill suited for physically-based rendering, shading, and
> *compositing*.”
>
>
> From my perspective PQ appears to have been designed with the cinema
> workflow in mind. That is, the final image is rendered, or “graded”,
> manually (by a colourist), in a reference cinema environment. The idea is
> that the movie will be seen in the same reference environment and, if this
> is true, the final grading is WYSIWYG. Unfortunately this isn’t always the
> case because movies are not always seen in a dark environment at the same
> brightness. For TV distribution (where the viewing environment is much
> brighter) the studios go to enormous trouble re-grading. Furthermore for 3D
> theatrical releases, where the display luminance is significantly less than
> 2D, the signal has to be separately graded. This leads, in part, to the
> enormous number of versions of each movie held by studios since, because
> each is display referred, there has to be a different version for each
> display and viewing environment.
>
>
> SDR and HLG, by contrast, are scene referred, which means that the signal
> does not imply any specific display or viewing environment. This is
> obviously important for the web, where there is no control of the viewing
> device or environment. Scene referred signals may be rendered for a wide
> range of display brightnesses and environments. This rendering is specified
> (for a reference viewing environment) in BT.2100. Extended range rendering,
> taking into account the effect of surrounding luminance, is described in
> "Display of high dynamic range images under varying viewing conditions"
> (full citation below).
>
>
> Craig Todd suggests that SDR, like PQ, is display referred. This is not
> the case. He mentions BT.1886, which specifies the EOTF (the display
> non-linearity, or gamma) for SDR. He rightly says that this was only
> belatedly specified, actually only in 2011. Whilst a specification for the
> assumed CRT characteristic was welcome, it was only specified 18 years
> after BT.709 (and a full 29(!) years after the standard definition
> specification, BT.601, which defines the same camera non-linearity as
> BT.709). This does rather suggest that it is less important than the camera
> (gamma correction) characteristic specified decades earlier. And this is
> because BT.601 and BT.709 specify the signal, as captured by the camera
> (not the way it should be rendered on a reference monitor). It is worth
> noting that BT.1886 does NOT specify a display brightness. Rather the
> brightness is scaled to the actual display brightness. PQ, on the other
> hand, explicitly specifies the brightness of a pixel on the (reference)
> display. So SDR signals are dimensionless, whereas PQ signals have
> dimensions of cd/m^2. All this makes it clear that PQ and SDR/HLG are
> fundamentally different.
>
>
> Craig further suggests that BT.2035 is somehow part of the SDR signal
> specification. This is clearly not the case having only been approved 31
> years after BT.601.
>
>
> So what does happen in practice with SDR? Viewers do indeed watch on
> brighter displays, in brighter environments, than the reference
> environment. However the signal is not simply stretched. It is re-rendered
> to suit the display and (assumed) viewing environment. TV manufacturers
> often include their own “secret sauce” to make the programs look “good”.
> But, typically, TV display gamma is 2.2, not the 2.4 specified in BT.1886,
> presumably because this make the pictures look psychovisually correct on
> brighter displays in brighter environments.
>
>
> It is correct that PQ is likely to be displayed at a higher luminance than
> is specified for a reference display. This is a bad thing. It requires that
> PQ is re-rendered, not merely stretched in brightness to match the display.
> Simply stretching the luminance results in poor quality pictures because a
> psychovisual adjustment is needed. Simply stretching the luminance makes
> the mid tones (e.g. flesh tones) look too bright and “misty” or “washed
> out”. Unfortunately PQ, unlike HLG, does not specify how to re-render PQ
> signals on a brighter display to make them look correct. For content
> producers, such as studios, this is a problem because they do not know how
> any specific TV or monitor will implement the display rendering. Their
> careful grading may be spoilt by unspecified display rendering.
>
>
> A further problem with re-rendering PQ is that it makes the effects of
> quantisation (“banding”) worse. PQ stands for perceptual quantisation and
> is based on setting quantisation levels so that quantisation can barely be
> seen on a reference display. This allows it to maximise dynamic range on
> that reference display. But, if you re-render PQ, you upset this carefully
> optimised quantisation strategy. Quantisation levels that were invisible on
> a reference display are stretched apart so that banding becomes visible on
> the brighter picture.
>
>
> So there are two serious problems in re-rendering PQ for a brighter
> display. Firstly no algorithm is standardised to do this. Hence producers
> cannot know how their carefully produced and graded content will appear to
> the final viewer. Secondly in stretching the luminance the effects of
> quantisation become more visible, thereby degrading the dynamic range. In
> contrast, rendering HLG on brighter displays is specified in BT.2100 and
> the effects of banding on HLG are (slightly) reduced as displays get
> brighter.
>
>
> The reason for these differences between PQ and HLG is due to the
> differences in their philosophy.
>
>
>  PQ appears to have been designed to try to produce the ultimate quality
> in a controlled, dark environment (such as a home cinema), whether it
> actually achieves this is a moot point. By assuming that PQ signals will be
> displayed in a controlled environment it is free to use increases in
> display brightness to produce brighter and brighter highlights. Since
> diffuse white is (now) set at 203 cd/m^2 it is, theoretically, possible for
> a PQ signal to have highlights up to about 50 times brighter than the
> diffuse part of the picture. Whether this is actually worth the bother is
> another moot point.
>
>
> HLG was designed to be a practical TV system. With TV, and on the web, we
> do not know what the display characteristics will be and what the ambient
> conditions will be. With TV and the web you certainly cannot assume you
> have a controlled, dark, environment. In a bright environment you cannot
> see very dark parts of the picture because they are swamped by ambient
> light. So, with HLG, we assume that brighter displays will be used to show
> pictures in brighter environments (not used for ever increasing
> highlights). The dynamic range allocated to HLG highlights remains constant
> and the whole picture is made brighter, on brighter screens, so that it can
> be seen in brighter environments (such as offices, on the move, in bedrooms
> or living rooms). We think this makes good sense for the TV and web use
> cases. The rendering for brighter displays is specified (backed up by
> published experimental results), and quantisation effects are not adversely
> affected by rendering for a brighter display.
>
>
> PQ can produce stunning results in a controlled dark environment (and so
> can HLG). But HLG seems simpler and more appropriate for applications such
> as TV and the web when the viewing conditions are not known.
>
>
> Best regards,
> Tim
>
>
> Selan, J.  2012.  “Cinematic color: from your monitor to the big screen.”
> In ACM SIGGRAPH  (2012)  Courses (SIGGRAPH  '12).  ACM,  New  York,  NY,
> USA,  ,  Article  9  ,  54  pages. DOI=http://dx.doi.org/10.1145/
> 2343483.2343492.  Also  available  as  a  Visual  Effects  Society  Technology
> Committee White Paper from http://cinematiccolor.com/
>
>
> <http://cinematiccolor.com/>
>
> Tim Borer, "Display of high dynamic range images under varying viewing
> conditions", Proc. SPIE 10396, Applications of Digital Image Processing XL,
> 103960H (19 September 2017); doi: 10.1117/12.2274253;
> http://dx.doi.org/10.1117/12.2274253
>
>
> Thompson, S. “Evaluation of Required Adjustments for HDR Displays under
> Domestic Ambient Conditions”. 7th  IEEE  International  conference  on
> consumer  electronics  in  Berlin  (“ICCE-Berlin”).  3rd  -  6th  September
> (2017).
>
>
>
> Dr Tim Borer MA, MSc, PhD, CEng, MIET, SMIEEE, Fellow SMPTE
> Lead Engineer
> Immersive & Interactive Content
> BBC Research & Development
> BBC Centre House, 56 Wood Lane, London  W12 7SB <https://maps.google.com/?q=56+Wood+Lane,+London+W12+7SB&entry=gmail&source=g>
>
> On 21/11/2017 00:00, Todd, Craig wrote:
>
> This group was brought to my attention today. Nice discussion going on.
>
> I’ve a few points and comments re the discussion of mapping/converting SDR
> into PQ HDR.
>
>
>
> Fredrik Hubinette wrote:
>
> “SDR should be mapped into HDR as when played in a reference environment,
>
> the composited result should then be adapted to the viewing environment.”
>
> I agree.
>
>
>
> First, backing up a bit, there is the issue as to how PQ should be
> displayed, and the comments (that I find nonsensical) that there is some
> fundamental difference between how PQ (and its “absolute” mapping of code
> values to pixel brightness) and SDR work in practice.
>
>
>
> Consider SDR: the EOTF was (belatedly) documented in BT.1886
> <http://www.itu.int/rec/R-REC-BT.1886-0-201103-I/en>, with reference
> brightness documented (also belatedly) in BT.2035
> <http://www.itu.int/rec/R-REC-BT.2035-0-201307-I/en>. Two documents that
> basically say use gamma 2.4 with 100% at 100 nits. The Scope of BT.2035
> states:
>
>
>
> “This Recommendation prescribes a method allowing HDTV producers or
> broadcasters to establish a *reference viewing condition* for evaluation
> of HDTV program material or completed programmes that can provide
> repeatable results from one facility to another when viewing the same
> material. This includes the display device and the surrounding
> environment.” (my emphasis).
>
>
>
> BT.2100 <http://www.itu.int/rec/R-REC-BT.2100-1-201706-I/en> defines a
> reference viewing environment, as well as reference PQ EOTF. The intention
> is that they go together. There is no intention expressed that the
> reference PQ EOTF should be used in a non-reference viewing environment.
>
>
>
> So what happens in practice with SDR? People watch SDR video in brighter
> environments than specified in BT.2035, and they use brighter displays than
> specified in BT.2035.
>
>
>
> So what should happen in practice with PQ HDR? People will watch in
> brighter environments than specified in BT.2100, and they will use brighter
> pixels than specified in BT.2100. To my mind, there is no fundamental
> difference in philosophy or practice between SDR and PQ HDR. The only
> difference is that with SDR the curve and the scaling of the curve were
> specified in separate documents. We have learned how to write better
> specifications; with BT.2100 we tried to define a complete HDR TV system in
> a single document.
>
>
>
> In the limited amount of PQ content to date, we have seen mid-tone
> brightness similar to that of SDR. See: https://www.dolby.com/us/en/
> technologies/dolby-vision/operational-guidelines-for-pq.pdf. A version of
> this study is included in the new Report ITU-R BT.2408
> <http://www.itu.int/pub/R-REP-BT.2408-2017> (see Annex 1).
>
>
>
> If full-scale 100 nit SDR white is mapped into PQ at 300 nits, what will
> happen? (Hint: on a *PQ reference display* it will look like a *living
> room SDR display*.) Typically the 100 nit (on an SDR reference display)
> SDR white will display at say 300 nits on a consumer display intended for
> use in a bright room. If an HDR display is set in a mode for best results
> in a bright room, it may likewise display a 100 nit PQ pixel at 300 nits.
> For an SDRx3 => PQ mapped image, the SDR white would then display at 900
> nits on the living room PQ display, making SDR whites come out 3x brighter
> than HDR content. More important than white level would be mid-tone levels,
> especially the level of faces. We certainly don’t want SDR faces displaying
> 3x brighter than HDR faces. So far, we’ve seen faces in PQ at levels close
> to those for SDR (using reference monitors for both). It is possible that
> over time, with more experience and familiarity with HDR images, HDR levels
> may creep up. It is also apparent that broadcast content often employs
> higher levels than movie or dramatic content.
>
>
>
> There is an approved (but as yet unpublished) update to Report BT.2390-2
> <http://www.itu.int/pub/R-REP-BT.2390-2-2017> that has an added section
> on SDR->HDR mapping. An excerpt is below (note that this language was very
> carefully crafted).
>
>
>
> *“10.1.1  Display referred mapping of SDR into PQ*
>
> The following procedure may be followed to achieve consistent mid-tone
> luminance levels when mapping standard dynamic range content into PQ.
>
> Standard dynamic range ITU-R BT.2020 content should be mapped to PQ by
> applying the ITU-R BT.1886 display EOTF and then applying the PQ EOTF-1.
> For unity mapping the peak signal of standard dynamic range content should
> be set to 100 cd/m2 or 51% PQ.
>
> Unity mapping does not change the display of the SDR content (it will
> display on the PQ HDR reference monitor the same as it displayed on the
> reference SDR monitor). Thus, no OOTF adjustment of the SDR display light
> signal is necessary.
>
> If the SDR content is being inserted into HDR programming, and there is
> desire to more closely match the brightness of the HDR content, and that
> brightness is known, scaling can be done to bring up the brightness of the
> mapped SDR content. Scaling should be performed with care lest scaled SDR
> content, in particular skin tones, becomes brighter than in the HDR
> content.”
>
>
>
> Hope the above helps with understanding.
>
> Best regards,
>
>
>
> Craig Todd
> Sr. VP and CTO
> Dolby Laboratories, Inc.
> 1275 Market St.
> <https://maps.google.com/?q=1275+Market+St.+%0D+San+Francisco,+CA+94103,+USA&entry=gmail&source=g>
> San Francisco, CA 94103, USA
> T  415-558-0221 <(415)%20558-0221>  M 415 672-0221 <(415)%20672-0221>
> www.dolby.com |ct@dolby.com
>
>
>
>
>