Re: ISO 21496‐1:20XX Gain Maps

> On Nov 15, 2023, at 18:48, Chris Lilley <chris@w3.org> wrote:
> 
> Hi folks,
> 
> This message is being sent, off-list, to everyone subscribed to public-png@w3.org, i.e. all members of the PNG WG. This is because ISO TC 42 sent a liaison request [1] to "W3C" asking for comments on their Gain Map draft, but the actual draft spec says it cannot be shared publicly. I have asked [2] that it be shared publicly so we can discuss it. Pending their reply, so that everyone in the WG can see it, here it is. Please do not further redistribute.
> 
> Gain Maps are an (allegedly) efficient way to provide an SDR baseline image plus an HDR alternate image. The claim of storage efficiency has not been backed up by any measurements that I have seen.

It's in the white-paper <https://helpx.adobe.com/camera-raw/using/gain-map.html> published by Adobe:

> Example Sizes
> The following numbers are based on a preliminary study of 50 raw images from digital cameras of
> natural content (landscape photographs) converted to 2000-pixel (long edge) renditions, RGB, Display
> P3 color space. The numbers represent the average size (per photo).
> • 8-bit uncompressed SDR rendition: 12842 KB
> • 8-bit JPEG compressed SDR rendition (Photoshop quality 90): 1747 KB
> • 16-bit JXL compressed SDR rendition (cjxl -d 1): 662 KB
> • 16-bit JXL compressed Gain Map, ¼ resolution, RGB quality 1 (cjxl -d 1): 101 KB (15% of SDR JXL)
> • 16-bit JXL compressed Gain Map, ¼ resolution, RGB quality 2 (cjxl -d 2): 60 KB (9% of SDR JXL)
> • 16-bit JXL compressed Gain Map, ¼ resolution, RGB quality 3 (cjxl -d 3): 43 KB (7% of SDR JXL)

Apple has also been using the precursor of this for many years: https://developer.apple.com/documentation/appkit/images_and_pdf/applying_apple_hdr_effect_to_your_photos?language=objc
Compared to storing both the SDR and HDR image it's significantly smaller in size. It's of course not lossless or exactly the same, but it's a good trade-off compared to the alternatives.

Gain maps that allow you to go from SDR->HDR make a lot of sense when it comes to backwards compatibility.

But gain maps that allow you to go from HDR->SDR are also highly useful. They allow:
1. Imaging pipelines to provide their own "look" for how an HDR image should look as SDR. Local tone-mapping from HDR to SDR is non-trivial and highly subjective and pipelines may want to apply their own artistic effects.
2. A defined way of interpolating between HDR and SDR. Highly useful when your display does not have the headroom to fully display the HDR content.

Cheers,
//Leo

> 
> The draft spec has a placeholder for storing gain maps in AVIF, HEIF and JPEG but does not have one for PNG. I asked that they add one for PNG. The placeholders have no content, in the current draft. I have been working on a proposal for gain maps in PNG, since I saw the first demo.
> 
> I'm aware that Apple and Adobe are working on this and have seen their implementations demonstrated, and I am aware Chrome is adding viewing support. Mostly, JPEG is being used for these demos. It seems like this will be a widely deployed method to deliver end-of-pipeline HDR content, so I feel we need to start discussing how that would be stored inside PNG.
> 
>   https://gregbenzphotography.com/hdr/
> https://community.adobe.com/t5/photoshop-ecosystem-ideas/allow-full-user-control-of-the-sdr-rendition-in-an-hdr-gain-map-export/idi-p/14205440
>   https://helpx.adobe.com/camera-raw/using/gain-map.html
> 
> [1] https://github.com/w3c/PNG-spec/issues/366
> [2] https://github.com/w3c/PNG-spec/issues/366#issuecomment-1799932581
> 
> -- 
> Chris Lilley
> @svgeesus
> Technical Director @ W3C
> W3C Strategy Team, Core Web Design
> W3C Architecture & Technology Team, Core Web & Media
>