Re: [EXTERNAL] XMP in PNG

The dynamic tone-mapping is more about mapping the content brightness to a specific displays capabilities.

For instance:

  *   The content is graded on a 1,000nit display
  *   The consumer display is 600nits
  *   Do you clip the highlights?   …or ideally apply a specific highlight knee to the content to compress them instead of clipping
  *   HDR10 allows this in a static fashion for the duration of the content.  Dolby Vision and HDR10+ use dynamic mapping (per frame or scene-based).
  *   Once the video has been mapped, what happens to graphics overlays?  These need to be mapped in a similar fashion or pre-composited together and then metadata needs to be re-calculated (can get complicated).

What you’re discussing (if I’m translating correctly) is “eye-adaption” which is certainly applicable but a whole other deep subject 😊.

The HDR-10 metadata and dynamic metadata are stored in the HEVC stream, QuickTime and MXF Wrappers that are well documented.  For still graphics, we would only need the HDR-10 metadata.  For HLG, this is a work in progress.  For dynamic metadata, we would only need the metadata for animated graphics.

Does that make sense so far?

There are others in Dolby, Samsung, BBC that might have additional thoughts here.

Best,
Chris


From: Chris Blume (ProgramMax) <programmax@gmail.com>
Date: Tuesday, November 30, 2021 at 10:47 AM
To: Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com>
Cc: public-png@w3.org <public-png@w3.org>
Subject: Re: [EXTERNAL] XMP in PNG
I believe I partly understand.
I remember working on HDR on Mac and seeing a video play that affected the brightness of more than just the video. If I remember correctly, the entire display's brightness changed as the video's brightness changed.

For example, if a bright flash is displayed and the viewer's pupils close, the rest of the display will have to become more bright to compensate and remain approximately the same color to the viewer.

Is my understanding correct?
(Is this considered part of the viewing environment?)


If I am on the right track, I can understand why extra metadata is useful.
The average brightness changes throughout the animation. And so we might have this metadata before every frame of animation. Or perhaps every other frame or only when a significant brightness change occurs.

Would we want to include this average brightness as part of the standard? If so, we are specifying a standard schema, too.
(Is iTXt + XMP + schema overhead too heavyweight for a simple average brightness?)

On Tue, Nov 30, 2021 at 9:54 AM Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com<mailto:Chris.Seeger@nbcuni.com>> wrote:
There could be additional metadata for specific HDR transfer functions that describes specific tone mapping or specific “looks”.

In the case of animated PNG’s, I’m anticipating this could also include dynamic metadata for HDR that is used for tone mapping against overlaid video that also includes dynamic metadata (and therefore could change it’s average brightness level which would in turn affect the overlaid graphics).

-Chris

From: Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>>
Date: Tuesday, November 30, 2021 at 7:03 AM
To: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: [EXTERNAL] XMP in PNG
Solicited feedback:
Chris Seeger, are you allowed to tell us what extra metadata you foresee coming around the corner? It would help us plan.
Everyone, feel free to weigh in.


Thoughts:
In our last meeting, we discussed adding XMP to PNG.
In this email thread we can talk about how we would like to accomplish that.

The motivation is from Chris Seeger, who asked if extra metadata could be added to Exif. Apparently, Exif is rigid and will not allow extra metadata. Leonard Rosenthol proposed XMP as an extensible metadata solution.

Here is the latest Exif standard (translated to English): https://www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf<https://urldefense.com/v3/__https:/www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf__;!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrx6K1JA-w$>
Here is a link to purchase ISO 16684-1, the XMP standard: https://www.iso.org/standard/75163.html<https://urldefense.com/v3/__https:/www.iso.org/standard/75163.html__;!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSry98HW4Ig$>



During the meeting, Chris Lilley mentioned that XMP in PNG could go inside the "iTXt" chunk<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*11iTXt__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg$> [1] with an unregistered keyword "XML:com.adobe.xmp". This makes sense to me. Other keywords represent other known metadata such as copyright, author, title, creation time...
Perhaps we could add an official keyword for XMP.

However, the datastream placement of "iTXt" and similar chunks has no restriction, according to PNG spec part 5.6, Chunk ordering<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*5ChunkOrdering__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw$> [2]. If the XMP data itself could affect how the image is decoded, we would want to enforce that it appears before the "IDAT" chunk in the datastream. Similarly, perhaps XMP would require a different (un)safe-to-copy property.

If we need strict chunk ordering or different chunk properties, we might want XMP to appear in a new chunk.



I do not know enough about XMP or the future extra metadata that we might be adding. So I cannot make a judgement call about this. I'll spend some time reading about XMP.


Interestingly, PNG spec part 12.10, Text chunk processing<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*12Text-chunk-processing__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw$> [3] suggests scenarios where text chunks might want to appear before or after the "IDAT" chunk.

[1] https://www.w3.org/TR/2003/REC-PNG-20031110/#11iTXt<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*11iTXt__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg$>
[2] https://www.w3.org/TR/2003/REC-PNG-20031110/#5ChunkOrdering<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*5ChunkOrdering__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw$>
[3] https://www.w3.org/TR/2003/REC-PNG-20031110/#12Text-chunk-processing<https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*12Text-chunk-processing__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw$>