Re: ISO 21496‐1:20XX Gain Maps

> On Nov 15, 2023, at 23:43, Pierre-Anthony Lemieux <pal@sandflow.com> wrote:
> 
> On Wed, Nov 15, 2023 at 2:37 PM Leo Barnes <lbarnes@apple.com <mailto:lbarnes@apple.com>> wrote:
>> 
>> 
>> 
>> On Nov 15, 2023, at 22:53, Pierre-Anthony Lemieux <pal@sandflow.com> wrote:
>> 
>> On Wed, Nov 15, 2023 at 12:24 PM Leo Barnes <lbarnes@apple.com> wrote:
>> 
>> 
>> 
>> 
>> On Nov 15, 2023, at 20:47, Pierre-Anthony Lemieux <pal@sandflow.com> wrote:
>> 
>> Thanks for that data, although it mainly covers JPEG-compressed images.
>> 
>> 
>> I think I understand gainmap in the context of compatibility with JPEG
>> 1, which is ubiquitous and cannot support HDR natively.
>> 
>> I am not sure how this applies to PNG and other formats that can
>> support HDR natively.
>> 
>> 
>> Please see these points made earlier:
>> 
>> But gain maps that allow you to go from HDR->SDR are also highly useful. They allow:
>> 1. Imaging pipelines to provide their own "look" for how an HDR image should look as SDR. Local tone-mapping from HDR to SDR is non-trivial and highly subjective and pipelines may want to apply their own artistic effects.
>> 
>> 
>> This is surprising to me since tone mapping to SDR ultimately depends
>> on user preference, viewing conditions, display capabilities, night
>> mode, etc. Why bake a particular SDR render in the image?
>> 
>> 
>> Because the image creator believed this to be the best SDR render for the image. Or the one that best matched the style that he/she prefers.
>> From the point of view of a content creator, they may want to specifically boost shadows or highlights in certain parts of an image in a way that regular tone mapping algorithms simply can't or don't do.
>> From the point of view of a camera manufacturer, they may want all their images to have a certain consistent style when shown in SDR.
> 
> My point is that AFAIK tone mapping will happen no matter what,
> defeating any "look" that camera baked in based on its local
> conditions.

Not completely sure I understand what you're saying here. My point is that with current HDR images the file creator does not have any control over how tone mapping to SDR is done. With a 21496‐1 gain map they do. Current global tone mapping algorithms don't always make the best choices, so there are definitely cases where a file creator would want to make tweaks. If your point is that a display will potentially need to tone map even an SDR image if the display conditions require it, this is true. But at least it will be closer to what the file creator intended rather than if the display was forced to tone map all the way from HDR (and won't look any different to if the file creator provided you with a dedicated SDR file).

21496‐1 is also not limited to staying in the same gamut, so it also gives the file creator the ability to control how gamut mapping is done. Which is for instance very useful if you have a Rec.2020 PQ 10-bit AVIF image and you want to control how it should look when transcoded to an 8-bit sRGB JPEG.

If you have a HDR display I can recommend downloading the Adobe sample app and the sample files and playing around with them. There are both samples that go from SDR->HDR and HDR->SDR.

> 
> On a related topic, what is the difference between (i) base sdr image
> where 1.0 means "sdr white" + gain map and (ii) hdr image where 1.0
> means "sdr white"?

My color science terminology is not the best, so I hope I make sense here. :)
Not completely sure I understand your question. 21496‐1 provides a way for interpolating between a base image and a (reconstructed) alternate image. The main use-case (currently) is to provide both a SDR and a HDR representation.
Application of gain map happens in display referred linear extended space. If the base and alternate images have the same color primaries, 1.0 in the base means the same thing as 1.0 in the alternate.

Happy to chat more on this topic if I didn't answer your question!
Cheers,
//Leo

> 
>> 
>> 
>> 2. A defined way of interpolating between HDR and SDR. Highly useful when your display does not have the headroom to fully display the HDR content.
>> 
>> 
>> Same above.
>> 
>> 
>> This goes hand in hand with the above. Apart from that, there have been complaints that various implementations do tone mapping from HDR to SDR differently and that the results may be inconsistent between various systems and applications. With the gainmap you provide all the details needed to show the image as the author intended it. Both in HDR and SDR and anywhere in between.
>> 
>> Cheers,
>> //Leo
>> 
>> 
>> 
>> Cheers,
>> //Leo
>> 
>> 
>> On Wed, Nov 15, 2023 at 11:12 AM Chris Lilley <chris@w3.org> wrote:
>> 
>> 
>> Thanks for that data, although it mainly covers JPEG-compressed images.
>> 
>> I would be interested to see a comparison between, for example:
>> 
>> 8 bit PNG compressed SDR image plus 10 bit PNG compressed HDR image
>> 8 bit PNG compressed baseline SDR image containing un-resampled 16-bit gain map
>> 
>> both in terms of file sizes and also in terms of per-pixel error visualization in the reconstructed alternate image.
>> 
>> On 2023-11-15 14:03, Leo Barnes wrote:
>> 
>> The claim of storage efficiency has not been backed up by any measurements that I have seen.
>> 
>> 
>> It's in the white-paper published by Adobe:
>> 
>> --
>> Chris Lilley
>> @svgeesus
>> Technical Director @ W3C
>> W3C Strategy Team, Core Web Design
>> W3C Architecture & Technology Team, Core Web & Media