Re: Discuss HDR at CSS WG next week ?

On Sun, Sep 18, 2016 at 10:19 PM, Mark Watson <watsonm@netflix.com> wrote:

>
>
> On Sep 18, 2016, at 6:23 PM, Rik Cabanier <cabanier@gmail.com> wrote:
>
>
>
> On Sat, Sep 17, 2016 at 7:23 PM, Mark Watson <watsonm@netflix.com> wrote:
>
>>
>>
>> On Fri, Sep 16, 2016 at 12:22 PM, Rik Cabanier <cabanier@gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Friday, September 16, 2016, Mark Watson <watsonm@netflix.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 10:11 AM, Rik Cabanier <cabanier@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Sep 15, 2016 at 12:22 PM, Rik Cabanier <cabanier@gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 11:57 AM, Mark Watson <watsonm@netflix.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 11:25 AM, Rik Cabanier <cabanier@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 11:01 AM, Mark Watson <watsonm@netflix.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:41 AM, Rik Cabanier <cabanier@gmail.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 10:21 AM, Mark Watson <
>>>>>>>>>> watsonm@netflix.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 15, 2016 at 9:58 AM, Rik Cabanier <
>>>>>>>>>>> cabanier@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 15, 2016 at 7:35 AM, Mark Watson <
>>>>>>>>>>>> watsonm@netflix.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> > On Sep 14, 2016, at 10:05 PM, Rossen Atanassov <
>>>>>>>>>>>>> Rossen.Atanassov@microsoft.com> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >> On Wed, Sep 14, 2016 at 16:53:53, Mark Watson wrote:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> I have some use-cases and issues / questions related to
>>>>>>>>>>>>> High Dynamic Range
>>>>>>>>>>>>> >> graphics / images and how they could be supported in CSS. I
>>>>>>>>>>>>> wondered if this
>>>>>>>>>>>>> >> topic was or could be on the agenda for next week ?
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Are you referring to our TPAC agenda? If so, we don't have
>>>>>>>>>>>>> the topic scheduled and looking through everything we have it is doubtful
>>>>>>>>>>>>> we could get to it.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Could you summarize your proposal? Is this something
>>>>>>>>>>>>> proposed at WICG already?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, TPAC. I've raised the issue of HDR a few times on the CSS
>>>>>>>>>>>>> list,
>>>>>>>>>>>>> but with no real response. We had a breakout at TPAC last
>>>>>>>>>>>>> year, but no
>>>>>>>>>>>>> one from CSS attended. I imagine that is because people do not
>>>>>>>>>>>>> yet
>>>>>>>>>>>>> have real hardware / platform APIs with which to play with this
>>>>>>>>>>>>> functionality. Those will soon be available, so it seemed good
>>>>>>>>>>>>> timing
>>>>>>>>>>>>> to raise it again and explain the various problems.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I only have problems, not a proposal. The basic problem (as I
>>>>>>>>>>>>> understand it) is that with HDR displays, users are unlikely
>>>>>>>>>>>>> to want
>>>>>>>>>>>>> the peak white for regular sRGB to map to the peak luminance
>>>>>>>>>>>>> of the
>>>>>>>>>>>>> display. That would make desktops blindingly bright. So there
>>>>>>>>>>>>> needs to
>>>>>>>>>>>>> be a way for pages to signal when they are providing data in
>>>>>>>>>>>>> the sRGB
>>>>>>>>>>>>> luminance space (where peak white is what the user has set as a
>>>>>>>>>>>>> comfortable luminance for their desktop according to current
>>>>>>>>>>>>> ambient
>>>>>>>>>>>>> light) and when they are providing data in a different
>>>>>>>>>>>>> luminance
>>>>>>>>>>>>> space, where peak white is brighter (exactly how bright is one
>>>>>>>>>>>>> of he
>>>>>>>>>>>>> questions to be answered.)
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This is defined in the CSS color spec [1]. If no profile is
>>>>>>>>>>>> provided, peak white is peak sRGB white.
>>>>>>>>>>>> The fact that you get blindingly white is a browser bug. (I
>>>>>>>>>>>> believe only Safari does this right today)
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ​I think there is a mis-understanding here and that is why I
>>>>>>>>>>> think the issue deserves discussion.
>>>>>>>>>>>
>>>>>>>>>>> As far as I can tell, [1] is entirely about color gamut. Whilst
>>>>>>>>>>> there are many ways to identify colors, the specification does not seem to
>>>>>>>>>>> treat dynamic ​range. The section you reference regarding profiles,
>>>>>>>>>>> says "Others are more human-friendly to write and understand, *and
>>>>>>>>>>> are converted to an sRGB color by CSS*".
>>>>>>>>>>>
>>>>>>>>>>> sRGB peak white will appear at the brightness the user prefers
>>>>>>>>>>> for their environment (because most of the graphics they see are sRGB and
>>>>>>>>>>> they will adjust the brightness control on their display).
>>>>>>>>>>>
>>>>>>>>>>> My point is that for capable displays, it should be possible to
>>>>>>>>>>> make graphics brighter than that.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, you can by either providing an image with an embedded
>>>>>>>>>> profile that has a higher gamut than sRGB, or by specifying a CSS color [1]
>>>>>>>>>> with an HDR profile.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ​Could you give an example of how you would specify an HDR profile
>>>>>>>>> with CSS ?​ For example, if I wanted some white text that was twice as
>>>>>>>>> bright as sRGB(255,255,255) text ? I don't see that in the css-color
>>>>>>>>> specification.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Dean wrote a blog about this: https://webkit.org/blog/
>>>>>>>> 6682/improving-color-on-the-web/
>>>>>>>>
>>>>>>>
>>>>>>> ​This is great, but it is all about gamut alone.​
>>>>>>>
>>>>>>> I don't know what profile is twice as bright as sRGB, but
>>>>>>>> color(rec2020 255 255 255) would be brighter than rgb(255, 255, 255)
>>>>>>>>
>>>>>>>
>>>>>>> ​Why would you think that ?
>>>>>>>
>>>>>>> In video, at least, the Rec.2020 color space is used for both SDR
>>>>>>> and HDR, with a variety of transfer functions. I don't know if Rec.2020
>>>>>>> defines a default transfer function, but if it does it is likely a gamma
>>>>>>> function like sRGB, which is scene-referred.
>>>>>>>
>>>>>>> Someone has to decide, or signal, how the luminance range of that
>>>>>>> 2020 data relates to the luminance range of sRGB data. It's not implied by
>>>>>>> the color profile alone.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> Note that I am not talking about a wider gamut - that seems to be
>>>>>>>>> well covered - but a different dynamic range (i.e. a larger color volume,
>>>>>>>>> not just larger color gamut).
>>>>>>>>>
>>>>>>>>
>>>>>>>> What do you mean by "color volume". Are you talking about higher
>>>>>>>> bit depths for compositing?
>>>>>>>>
>>>>>>>
>>>>>>> ​No, but higher bit depths are important too.
>>>>>>>
>>>>>>> As far as I can tell, CSS and ICC profiles deal with color gamut:
>>>>>>> subspaces of the 2-dimensional CIE color space. A color in that space can
>>>>>>> be described by two coordinates.
>>>>>>>
>>>>>>
>>>>> This is incorrect. ICC profiles use 3 dimensions. Most charts show
>>>>> color spaces as 2D projections which is probably why you think this.
>>>>>
>>>>>
>>>>>> An actual pixel color has 3 coordinates and res​ides in a *color
>>>>>>> volume*: a three-dimensional space where the intersection of this
>>>>>>> space and a plane is the CIE space.
>>>>>>>
>>>>>>
>>>>> Color volume is part of ICC.
>>>>>
>>>>>
>>>>>> The problem we have can be thought of as a backwards compatibilty
>>>>>>> one. You *could* specify luminance on the same scale for all image
>>>>>>> / graphics data sources. And, indeed, I expect this is what the graphics
>>>>>>> plane APIs of all devices will look like. In that case an RGB pixel in any
>>>>>>> color space with linear values ( 1.0, 1.0, 1.0 ) will be rendered display
>>>>>>> peak luminance (or close). I believe this is the case with CSS as things
>>>>>>> stand.
>>>>>>>
>>>>>>
>>>>> According to the spec, rgb(255,255,255) is rendered on as peak sRGB
>>>>> luminance which isn't necessarily the peak of what your devices is capable
>>>>> of.
>>>>> This is why you can now specify a profile (which controls the
>>>>> brightness) so you can attain what your display is capable of.
>>>>>
>>>>>
>>>>>> This would support any and all HDR images / graphics, but the problem
>>>>>>> would be that peak white in both SDR and HDR data would be rendered the
>>>>>>> same. Most importantly, peak white in desktop graphics would be the same as
>>>>>>> specular highlights in HDR images.
>>>>>>>
>>>>>>
>>>>> No since peak white is controlled by the ICC profile.
>>>>>
>>>>>
>>>>>> Since most desktop applications are going to stay sRGB, we need some
>>>>>>> new way for authors to specify pixels that are brighter than that.
>>>>>>>
>>>>>>
>>>>> Why do you think that desktop apps are staying in sRGB? Most graphics
>>>>> applications support ICC today and the good ones support HDR.
>>>>>
>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>> Those colors are then mapped to the "device" color which usually
>>>>>>>>>> is the monitor's colorspace. If you have an HDR monitor and specify an HDR
>>>>>>>>>> image, you will get colors that are brighter than sRGB white.
>>>>>>>>>>
>>>>>>>>>> Safari is already doing this for images so you can try this for
>>>>>>>>>> yourself if you have a capable device.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> If you do provide a profile, it will be used to map to the
>>>>>>>>>>>> display.
>>>>>>>>>>>> Is there a way to embed a profile in a video stream?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ​Here I am not talking about video. But, yes, video metadata
>>>>>>>>>>> specifies not only color gamut (primaries) but a transfer function which is
>>>>>>>>>>> used to map between encoded luminance and actual luminance and is different
>>>>>>>>>>> for HDR and SDR video.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I think we need to specify what is done with video. It seems that
>>>>>>>>>> this information needs to be used so the browser knows how to manage the
>>>>>>>>>> video's colors.
>>>>>>>>>> what happens today with video?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ​Well, this is all rather new, so I doubt there is any
>>>>>>>>> consistency. On the various devices we have seen which composite sRGB
>>>>>>>>> graphics with HDR video it's highly variable. I expect when we see this on
>>>>>>>>> devices with desktop-class browsers, the operating system will have made
>>>>>>>>> some decision as to what luminance to map sRGB to, relative to the HDR
>>>>>>>>> images. I've heard 80 nits. I've also heard of people planning to map sRGB
>>>>>>>>> peak white to panel peak white.​
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> To obtain CSS use-cases, imagine first a still image extracted
>>>>>>>>>>> from an HDR video, encoded in some still image format which carries HDR
>>>>>>>>>>> metadata and which is supported by the browser.
>>>>>>>>>>>
>>>>>>>>>>> It should be possible to specify in CSS any color+luminance that
>>>>>>>>>>> appears in that image. If the video or still image pixels are constrained
>>>>>>>>>>> to be no brighter than the sRGB peak white (the luminance the user prefers
>>>>>>>>>>> for their desktop), then you will not get any improvement in appearance:
>>>>>>>>>>> for example specular highlights in the video / image would be no brighter
>>>>>>>>>>> than for an SDR video / image, even though the display may be capable of
>>>>>>>>>>> that.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, extract the transfer function and other information from the
>>>>>>>>>> video stream and use it to construct a profile. Then embed that profile in
>>>>>>>>>> your still image.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ​What about actually specifying one of those colors in CSS ? So
>>>>>>>>> that I can have text or graphics matching ​the colors in the image ?
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can place the profile on your web server,load it with css using
>>>>>>>> @color-profile and use the name with the css color property.
>>>>>>>>
>>>>>>>
>>>>>>> ​Like I said, I don't think the profile is enough, since it is
>>>>>>> addressing gamut.​ Certainly, Rec. 2020, for example, does not tell me how
>>>>>>> much brighter than sRGB it's peak white should be. In video we have
>>>>>>> additional metadata for that.
>>>>>>>
>>>>>>
>>>>>> It seems that there are some differences between the video and
>>>>>> desktop world.
>>>>>> I'm going to talk to some of our color experts to learn more about
>>>>>> this subject.
>>>>>>
>>>>>
>>>>> I talked to our people that deal with color for our video application
>>>>> (premiere and after effects) and they say there's no difference between
>>>>> video and desktop profiles.
>>>>>
>>>>
>>>> In video, signals are accompanied with data including, separately, the
>>>> color gamut and the transfer function. If I look at BT.2020, for example,
>>>> it specifies color primaries and a scene-referred transfer function mapping
>>>> ​the quantized tristimulus values to linear in the range 0-1. BT.2020 does
>>>> not tell you "how bright" (1,1,1) should be.
>>>>
>>>> It does say:
>>>>
>>>> "In typical production practice the encoding function of image sources
>>>> is adjusted so that the final picture has the desired look, as viewed on a
>>>> reference monitor having the reference decoding function of Recommendation
>>>> ITU-R BT.1886, in the reference viewing environment defined in
>>>> Recommendation ITU-R BT.2035."
>>>>
>>>> and BT.1886 specifies a peak luminance of 100 nits (the standard for
>>>> SDR TV).
>>>>
>>>> For HDR video, we use the BT.2020 color gamut (primaries), with the
>>>> SMPTE ST.2084 Transfer Function (PQ) which specifies mapping from quantized
>>>> tristimulus values to absolute luminance (from 0 to 10,000nits).
>>>>
>>>> So, I still have the problem that specifying "rec2020" or "dci-p3" in
>>>> CSS does not tell me how the luminance of those pixel values should be
>>>> rendered relative to the luminance of "srgb". Indeed, if I look at Section
>>>> 10.2, it specifies the mappings associated with these color spaces to a
>>>> linear [ 0, 1 ] space. A natural assumption is that all three have the same
>>>> peak luminance (whatever you map 1.0 to, determined by the brightness
>>>> control on the display).
>>>>
>>>> I do see that ICC profiles can include a luminanceTag field, specifying
>>>> "absolute luminance of emissive devices in candelas per square metre as
>>>> described by the Y channel". So perhaps what I am looking for is a variant
>>>> of the "rec2020" profile with this luminanceTag set to 10,000 and somehow
>>>> specifying the PQ transfer function instead of the one in BT.2020.
>>>>
>>>>
>>>>
>>> I'm unsure about the video case. For images and artwork, all the
>>> information you need should be covered by the ICC workflow.
>>>
>>> If you want, I can put you in touch with one of our color experts since
>>> they are more familiar with the details.
>>>
>>
>> ​Yes, that would be great, thanks. But I would also like an opportunity
>> to discuss the problem with the wider CSS group. At a minimum, even if
>> profiles solve this, the specification is unclear as to what the rec2020
>> profile refers to, since the 2020 colorspace is used for both SDR and HDR
>> in video.​
>>
>
> I got a reply from our color people:
>
> [snip]
>
> iMac now supports HDR, with a peak of 500 nits.
>
> Classic SDR content is not mapped to 500 nits.
> Instead it maps to a brightness set by the user, often around 200 nits.
> Classic SDR content includes all legacy apps, such as Safari, MS Word,
> email…
> For these apps, MacOS maps their diffuse white (1.0, 255 etc)  to this 200
> nit setting.
>
> In order to display HDR content above 200 nits, an app must produce frame
> buffer values > 1.0.
> Classic apps don’t know how to do that.
>
> HDR aware apps know that the peak of all non-HDR color spaces (sRGB,
> BT.2020…) should be mapped to 1.0, which OS maps to 200 nits.
> And for HDR spaces, they know that the diffuse white point in the HDR
> space is also mapped to 1.0.
> The diffuse white point for PQ-encoding is at 100-200 nits, not max code
> value.
> AFAIK both Apple and Msft maps PQ for 100 nits to diffuse white.
>
> If you want something brighter than sRGB white, then use a proper HDR
> color space.
> BT.2020 is not an HDR space.
> HDR10 is an HDR space, with same primaries as BT.2020.
>
>
> If you combine this with Simon Fraser's email:
>
> Compositing happens in 10-bit or half-float textures which may contain < 0
> or > 1 values. A P3 image is mapped into this unclipped sRGB space for
> compositing, then the final result is mapped into Display-P3 at the end.
>
> then it looks that if you can tag something with HDR10 and the color
> values are high enough, the result will be brighter than than 200 nits at
> least on system that support HDR. (The unclipped sRGB space does sound a
> bit weird.)
>
>>
>>
>>> I just use profiles as a black box. 😀
>>>
>>
>> ​The full power of iCC profiles may solve this. But what seems unclear to
>> me is whether the profiles defined in CSS (sRGB, rec2020, dci-p3) are well
>> enough defined for it to be clear how their luminance ranges should be
>> mapped to ​actual physical luminance, *relative to each other*.
>>
>> For example, whilst sRGB, at least as defined in [1], defines peak white
>> to be 80nits, BT.2020 (as I noted above) does not specify a peak luminance
>> and at least with video is used with different transfer functions with peak
>> luminance 100nits (SDR) or 10,000nits (HDR with PQ). So, what does it mean
>> when you specify rec2020 in CSS ? And, if it means 100nits, because 2020
>> notes that this is a "typical" usage, how do I specify a profile where 2020
>> colorspace is used with 10,000nits peak ? Do implementations *actually*
>> map the 2020 luminance range to 125% of the sRGB range and is that even the
>> right thing to do ?
>>
>> (and this is before we even start talking about how video metadata can
>> influence tone mapping and how one might be able to get the same tone
>> mapping applied to graphics / images).
>>
>> If there is not time on the CSS agenda (which is see is rather full) for
>> this, I wonder if there would be interest in a break-out session on
>> Wednesday ?
>>
>
> We could. The hard part would be to track down the experts. It's easy to
> get confused by all the different color standards.
> Your questions seem more about quality of implementation than the w3c
> color specification.
>
>
> I think your answer above demonstrates that there is a spec issue, not
> just quality-of-implementation issues, since it is by no means clear from
> the specification that the approach taken by Apple is the only one.
>

Currently, the spec chooses not to specify how a document should be
rendered. This could be added once implementations and other specs (such as
ICC) progress.

However, if you follow the proposed color spec and tag everything the way
you'd like it to render, the ICC conversions will give you the best output
per the current output device.



> The approach described above - where the value 1.0 always means the same
> thing independent of color space, seems reasonable, but also it would be
> reasonable that if a color is tagged as being from an HDR space that the
> peak white of that space is also 1.0.
>

Yes, that would be nice.


> The question of what luminance value in an HDR space should be mapped to
> the same as sRGB diffuse white (the 1.0 value) is not answered in the HDR
> specifications I am aware of.
>
> Also, it would be nice to have a shorthand for 2020 color primaries used
> with an HDR transfer function, for example as defined in BT.2100 with PQ.
>

These 3 issues are about HDR ICC profiles; not with the CSS color spec or
anything that is controlled by w3c.
Maybe you should take this up with that organization?


> These issues at least deserve discussion.
>

Yes. Let's have a meeting on Wednesday morning. Leonard is familiar with
this ares and agreed to attend.


>
>> ​[1] https://www.w3.org/Graphics/Color/srgb​
>>
>>
>>
>>>
>>> If more is needed than what is already specked, we can bring it back to
>>> the list.
>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>> I will be at TPAC next week as well so maybe we can have a short
>>>>>> get-together.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> If the WICG is the appropriate place to raise these problems,
>>>>>>>>>>>>> I can do
>>>>>>>>>>>>> that, but the experts are in CSS WG so I wondered if there was
>>>>>>>>>>>>> interest in learning about these issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks ... Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1: https://drafts.csswg.org/css-color/#color-type
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> 1: https://drafts.csswg.org/css-color/#icc-colors
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> 1: https://drafts.csswg.org/css-color/#at-profile
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>