Re: Discuss HDR at CSS WG next week ?

On Friday, September 16, 2016, Mark Watson <watsonm@netflix.com> wrote:

>
>
> On Fri, Sep 16, 2016 at 10:11 AM, Rik Cabanier <cabanier@gmail.com
> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>
>>
>>
>> On Thu, Sep 15, 2016 at 12:22 PM, Rik Cabanier <cabanier@gmail.com
>> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>>
>>>
>>>
>>> On Thu, Sep 15, 2016 at 11:57 AM, Mark Watson <watsonm@netflix.com
>>> <javascript:_e(%7B%7D,'cvml','watsonm@netflix.com');>> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Sep 15, 2016 at 11:25 AM, Rik Cabanier <cabanier@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Sep 15, 2016 at 11:01 AM, Mark Watson <watsonm@netflix.com
>>>>> <javascript:_e(%7B%7D,'cvml','watsonm@netflix.com');>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 10:41 AM, Rik Cabanier <cabanier@gmail.com
>>>>>> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 10:21 AM, Mark Watson <watsonm@netflix.com
>>>>>>> <javascript:_e(%7B%7D,'cvml','watsonm@netflix.com');>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 9:58 AM, Rik Cabanier <cabanier@gmail.com
>>>>>>>> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 7:35 AM, Mark Watson <watsonm@netflix.com
>>>>>>>>> <javascript:_e(%7B%7D,'cvml','watsonm@netflix.com');>> wrote:
>>>>>>>>>
>>>>>>>>>> > On Sep 14, 2016, at 10:05 PM, Rossen Atanassov <
>>>>>>>>>> Rossen.Atanassov@microsoft.com
>>>>>>>>>> <javascript:_e(%7B%7D,'cvml','Rossen.Atanassov@microsoft.com');>>
>>>>>>>>>> wrote:
>>>>>>>>>> >
>>>>>>>>>> >> On Wed, Sep 14, 2016 at 16:53:53, Mark Watson wrote:
>>>>>>>>>> >>
>>>>>>>>>> >> I have some use-cases and issues / questions related to High
>>>>>>>>>> Dynamic Range
>>>>>>>>>> >> graphics / images and how they could be supported in CSS. I
>>>>>>>>>> wondered if this
>>>>>>>>>> >> topic was or could be on the agenda for next week ?
>>>>>>>>>> >
>>>>>>>>>> > Are you referring to our TPAC agenda? If so, we don't have the
>>>>>>>>>> topic scheduled and looking through everything we have it is doubtful we
>>>>>>>>>> could get to it.
>>>>>>>>>> >
>>>>>>>>>> > Could you summarize your proposal? Is this something proposed
>>>>>>>>>> at WICG already?
>>>>>>>>>>
>>>>>>>>>> Yes, TPAC. I've raised the issue of HDR a few times on the CSS
>>>>>>>>>> list,
>>>>>>>>>> but with no real response. We had a breakout at TPAC last year,
>>>>>>>>>> but no
>>>>>>>>>> one from CSS attended. I imagine that is because people do not yet
>>>>>>>>>> have real hardware / platform APIs with which to play with this
>>>>>>>>>> functionality. Those will soon be available, so it seemed good
>>>>>>>>>> timing
>>>>>>>>>> to raise it again and explain the various problems.
>>>>>>>>>>
>>>>>>>>>> I only have problems, not a proposal. The basic problem (as I
>>>>>>>>>> understand it) is that with HDR displays, users are unlikely to
>>>>>>>>>> want
>>>>>>>>>> the peak white for regular sRGB to map to the peak luminance of
>>>>>>>>>> the
>>>>>>>>>> display. That would make desktops blindingly bright. So there
>>>>>>>>>> needs to
>>>>>>>>>> be a way for pages to signal when they are providing data in the
>>>>>>>>>> sRGB
>>>>>>>>>> luminance space (where peak white is what the user has set as a
>>>>>>>>>> comfortable luminance for their desktop according to current
>>>>>>>>>> ambient
>>>>>>>>>> light) and when they are providing data in a different luminance
>>>>>>>>>> space, where peak white is brighter (exactly how bright is one of
>>>>>>>>>> he
>>>>>>>>>> questions to be answered.)
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This is defined in the CSS color spec [1]. If no profile is
>>>>>>>>> provided, peak white is peak sRGB white.
>>>>>>>>> The fact that you get blindingly white is a browser bug. (I
>>>>>>>>> believe only Safari does this right today)
>>>>>>>>>
>>>>>>>>
>>>>>>>> ​I think there is a mis-understanding here and that is why I think
>>>>>>>> the issue deserves discussion.
>>>>>>>>
>>>>>>>> As far as I can tell, [1] is entirely about color gamut. Whilst
>>>>>>>> there are many ways to identify colors, the specification does not seem to
>>>>>>>> treat dynamic ​range. The section you reference regarding profiles,
>>>>>>>> says "Others are more human-friendly to write and understand, *and
>>>>>>>> are converted to an sRGB color by CSS*".
>>>>>>>>
>>>>>>>> sRGB peak white will appear at the brightness the user prefers for
>>>>>>>> their environment (because most of the graphics they see are sRGB and they
>>>>>>>> will adjust the brightness control on their display).
>>>>>>>>
>>>>>>>> My point is that for capable displays, it should be possible to
>>>>>>>> make graphics brighter than that.
>>>>>>>>
>>>>>>>
>>>>>>> Yes, you can by either providing an image with an embedded profile
>>>>>>> that has a higher gamut than sRGB, or by specifying a CSS color [1] with an
>>>>>>> HDR profile.
>>>>>>>
>>>>>>
>>>>>> ​Could you give an example of how you would specify an HDR profile
>>>>>> with CSS ?​ For example, if I wanted some white text that was twice as
>>>>>> bright as sRGB(255,255,255) text ? I don't see that in the css-color
>>>>>> specification.
>>>>>>
>>>>>
>>>>> Dean wrote a blog about this: https://webkit.org/blog/
>>>>> 6682/improving-color-on-the-web/
>>>>>
>>>>
>>>> ​This is great, but it is all about gamut alone.​
>>>>
>>>> I don't know what profile is twice as bright as sRGB, but color(rec2020
>>>>> 255 255 255) would be brighter than rgb(255, 255, 255)
>>>>>
>>>>
>>>> ​Why would you think that ?
>>>>
>>>> In video, at least, the Rec.2020 color space is used for both SDR and
>>>> HDR, with a variety of transfer functions. I don't know if Rec.2020 defines
>>>> a default transfer function, but if it does it is likely a gamma function
>>>> like sRGB, which is scene-referred.
>>>>
>>>> Someone has to decide, or signal, how the luminance range of that 2020
>>>> data relates to the luminance range of sRGB data. It's not implied by the
>>>> color profile alone.
>>>>
>>>>
>>>>>
>>>>>> Note that I am not talking about a wider gamut - that seems to be
>>>>>> well covered - but a different dynamic range (i.e. a larger color volume,
>>>>>> not just larger color gamut).
>>>>>>
>>>>>
>>>>> What do you mean by "color volume". Are you talking about higher bit
>>>>> depths for compositing?
>>>>>
>>>>
>>>> ​No, but higher bit depths are important too.
>>>>
>>>> As far as I can tell, CSS and ICC profiles deal with color gamut:
>>>> subspaces of the 2-dimensional CIE color space. A color in that space can
>>>> be described by two coordinates.
>>>>
>>>
>> This is incorrect. ICC profiles use 3 dimensions. Most charts show color
>> spaces as 2D projections which is probably why you think this.
>>
>>
>>> An actual pixel color has 3 coordinates and res​ides in a *color volume*:
>>>> a three-dimensional space where the intersection of this space and a plane
>>>> is the CIE space.
>>>>
>>>
>> Color volume is part of ICC.
>>
>>
>>> The problem we have can be thought of as a backwards compatibilty one.
>>>> You *could* specify luminance on the same scale for all image /
>>>> graphics data sources. And, indeed, I expect this is what the graphics
>>>> plane APIs of all devices will look like. In that case an RGB pixel in any
>>>> color space with linear values ( 1.0, 1.0, 1.0 ) will be rendered display
>>>> peak luminance (or close). I believe this is the case with CSS as things
>>>> stand.
>>>>
>>>
>> According to the spec, rgb(255,255,255) is rendered on as peak sRGB
>> luminance which isn't necessarily the peak of what your devices is capable
>> of.
>> This is why you can now specify a profile (which controls the brightness)
>> so you can attain what your display is capable of.
>>
>>
>>> This would support any and all HDR images / graphics, but the problem
>>>> would be that peak white in both SDR and HDR data would be rendered the
>>>> same. Most importantly, peak white in desktop graphics would be the same as
>>>> specular highlights in HDR images.
>>>>
>>>
>> No since peak white is controlled by the ICC profile.
>>
>>
>>> Since most desktop applications are going to stay sRGB, we need some new
>>>> way for authors to specify pixels that are brighter than that.
>>>>
>>>
>> Why do you think that desktop apps are staying in sRGB? Most graphics
>> applications support ICC today and the good ones support HDR.
>>
>>
>>>
>>>>>
>>>>>> Those colors are then mapped to the "device" color which usually is
>>>>>>> the monitor's colorspace. If you have an HDR monitor and specify an HDR
>>>>>>> image, you will get colors that are brighter than sRGB white.
>>>>>>>
>>>>>>> Safari is already doing this for images so you can try this for
>>>>>>> yourself if you have a capable device.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> If you do provide a profile, it will be used to map to the display.
>>>>>>>>> Is there a way to embed a profile in a video stream?
>>>>>>>>>
>>>>>>>>
>>>>>>>> ​Here I am not talking about video. But, yes, video metadata
>>>>>>>> specifies not only color gamut (primaries) but a transfer function which is
>>>>>>>> used to map between encoded luminance and actual luminance and is different
>>>>>>>> for HDR and SDR video.
>>>>>>>>
>>>>>>>
>>>>>>> I think we need to specify what is done with video. It seems that
>>>>>>> this information needs to be used so the browser knows how to manage the
>>>>>>> video's colors.
>>>>>>> what happens today with video?
>>>>>>>
>>>>>>
>>>>>> ​Well, this is all rather new, so I doubt there is any consistency.
>>>>>> On the various devices we have seen which composite sRGB graphics with HDR
>>>>>> video it's highly variable. I expect when we see this on devices with
>>>>>> desktop-class browsers, the operating system will have made some decision
>>>>>> as to what luminance to map sRGB to, relative to the HDR images. I've heard
>>>>>> 80 nits. I've also heard of people planning to map sRGB peak white to panel
>>>>>> peak white.​
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> To obtain CSS use-cases, imagine first a still image extracted from
>>>>>>>> an HDR video, encoded in some still image format which carries HDR metadata
>>>>>>>> and which is supported by the browser.
>>>>>>>>
>>>>>>>> It should be possible to specify in CSS any color+luminance that
>>>>>>>> appears in that image. If the video or still image pixels are constrained
>>>>>>>> to be no brighter than the sRGB peak white (the luminance the user prefers
>>>>>>>> for their desktop), then you will not get any improvement in appearance:
>>>>>>>> for example specular highlights in the video / image would be no brighter
>>>>>>>> than for an SDR video / image, even though the display may be capable of
>>>>>>>> that.
>>>>>>>>
>>>>>>>
>>>>>>> Yes, extract the transfer function and other information from the
>>>>>>> video stream and use it to construct a profile. Then embed that profile in
>>>>>>> your still image.
>>>>>>>
>>>>>>
>>>>>> ​What about actually specifying one of those colors in CSS ? So that
>>>>>> I can have text or graphics matching ​the colors in the image ?
>>>>>>
>>>>>
>>>>> You can place the profile on your web server,load it with css using
>>>>> @color-profile and use the name with the css color property.
>>>>>
>>>>
>>>> ​Like I said, I don't think the profile is enough, since it is
>>>> addressing gamut.​ Certainly, Rec. 2020, for example, does not tell me how
>>>> much brighter than sRGB it's peak white should be. In video we have
>>>> additional metadata for that.
>>>>
>>>
>>> It seems that there are some differences between the video and desktop
>>> world.
>>> I'm going to talk to some of our color experts to learn more about this
>>> subject.
>>>
>>
>> I talked to our people that deal with color for our video application
>> (premiere and after effects) and they say there's no difference between
>> video and desktop profiles.
>>
>
> In video, signals are accompanied with data including, separately, the
> color gamut and the transfer function. If I look at BT.2020, for example,
> it specifies color primaries and a scene-referred transfer function mapping
> ​the quantized tristimulus values to linear in the range 0-1. BT.2020 does
> not tell you "how bright" (1,1,1) should be.
>
> It does say:
>
> "In typical production practice the encoding function of image sources is
> adjusted so that the final picture has the desired look, as viewed on a
> reference monitor having the reference decoding function of Recommendation
> ITU-R BT.1886, in the reference viewing environment defined in
> Recommendation ITU-R BT.2035."
>
> and BT.1886 specifies a peak luminance of 100 nits (the standard for SDR
> TV).
>
> For HDR video, we use the BT.2020 color gamut (primaries), with the SMPTE
> ST.2084 Transfer Function (PQ) which specifies mapping from quantized
> tristimulus values to absolute luminance (from 0 to 10,000nits).
>
> So, I still have the problem that specifying "rec2020" or "dci-p3" in CSS
> does not tell me how the luminance of those pixel values should be rendered
> relative to the luminance of "srgb". Indeed, if I look at Section 10.2, it
> specifies the mappings associated with these color spaces to a linear [ 0,
> 1 ] space. A natural assumption is that all three have the same peak
> luminance (whatever you map 1.0 to, determined by the brightness control on
> the display).
>
> I do see that ICC profiles can include a luminanceTag field, specifying
> "absolute luminance of emissive devices in candelas per square metre as
> described by the Y channel". So perhaps what I am looking for is a variant
> of the "rec2020" profile with this luminanceTag set to 10,000 and somehow
> specifying the PQ transfer function instead of the one in BT.2020.
>
>
>
I'm unsure about the video case. For images and artwork, all the
information you need should be covered by the ICC workflow.

If you want, I can put you in touch with one of our color experts since
they are more familiar with the details. I just use profiles as a black
box. 😀

If more is needed than what is already specked, we can bring it back to the
list.

>
>
>
>>
>>
>>> I will be at TPAC next week as well so maybe we can have a short
>>> get-together.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> If the WICG is the appropriate place to raise these problems, I
>>>>>>>>>> can do
>>>>>>>>>> that, but the experts are in CSS WG so I wondered if there was
>>>>>>>>>> interest in learning about these issues.
>>>>>>>>>>
>>>>>>>>>> Thanks ... Mark
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1: https://drafts.csswg.org/css-color/#color-type
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> 1: https://drafts.csswg.org/css-color/#icc-colors
>>>>>>>
>>>>>>>
>>>>>>
>>>>> 1: https://drafts.csswg.org/css-color/#at-profile
>>>>>
>>>>>
>>>>
>>>
>>
>