Re: [EXTERNAL] XMP in PNG

Perhaps correcting myself:
Rather than metadata providing average brightness, it could provide an
array of tone mapping options based on the OTHER displayed content's
average brightness. And thus it would be a 1-time chunk, not a per-frame
chunk.

Or maybe it is a combination of the two.

(A decoder could track average brightness easily enough for still images.
Or for images where a full frame update happens. It just tracks it while it
decodes. However, since APNG could update just a subregion of the image
this is more difficult. Providing per-frame average brightness could be
useful.)

On Tue, Nov 30, 2021 at 10:46 AM Chris Blume (ProgramMax) <
programmax@gmail.com> wrote:

> I believe I partly understand.
> I remember working on HDR on Mac and seeing a video play that affected the
> brightness of more than just the video. If I remember correctly, the entire
> display's brightness changed as the video's brightness changed.
>
> For example, if a bright flash is displayed and the viewer's pupils close,
> the rest of the display will have to become more bright to compensate and
> remain approximately the same color to the viewer.
>
> Is my understanding correct?
> (Is this considered part of the viewing environment?)
>
>
> If I am on the right track, I can understand why extra metadata is useful.
> The average brightness changes throughout the animation. And so we might
> have this metadata before every frame of animation. Or perhaps every other
> frame or only when a significant brightness change occurs.
>
> Would we want to include this average brightness as part of the standard?
> If so, we are specifying a standard schema, too.
> (Is iTXt + XMP + schema overhead too heavyweight for a simple average
> brightness?)
>
> On Tue, Nov 30, 2021 at 9:54 AM Seeger, Chris (NBCUniversal) <
> Chris.Seeger@nbcuni.com> wrote:
>
>> There could be additional metadata for specific HDR transfer functions
>> that describes specific tone mapping or specific “looks”.
>>
>>
>>
>> In the case of animated PNG’s, I’m anticipating this could also include
>> dynamic metadata for HDR that is used for tone mapping against overlaid
>> video that also includes dynamic metadata (and therefore could change it’s
>> average brightness level which would in turn affect the overlaid graphics).
>>
>>
>>
>> -Chris
>>
>>
>>
>> *From: *Chris Blume (ProgramMax) <programmax@gmail.com>
>> *Date: *Tuesday, November 30, 2021 at 7:03 AM
>> *To: *public-png@w3.org <public-png@w3.org>
>> *Subject: *[EXTERNAL] XMP in PNG
>>
>> *Solicited feedback:*
>> Chris Seeger, are you allowed to tell us what extra metadata you foresee
>> coming around the corner? It would help us plan.
>> Everyone, feel free to weigh in.
>>
>>
>> *Thoughts:*
>> In our last meeting, we discussed adding XMP to PNG.
>> In this email thread we can talk about how we would like to accomplish
>> that.
>>
>> The motivation is from Chris Seeger, who asked if extra metadata could be
>> added to Exif. Apparently, Exif is rigid and will not allow extra metadata.
>> Leonard Rosenthol proposed XMP as an extensible metadata solution.
>>
>> Here is the latest Exif standard (translated to English):
>> https://www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf
>> <https://urldefense.com/v3/__https:/www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf__;!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrx6K1JA-w$>
>> Here is a link to purchase ISO 16684-1, the XMP standard:
>> https://www.iso.org/standard/75163.html
>> <https://urldefense.com/v3/__https:/www.iso.org/standard/75163.html__;!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSry98HW4Ig$>
>>
>>
>>
>>
>>
>>
>>
>> During the meeting, Chris Lilley mentioned that XMP in PNG could go
>> inside the "iTXt" chunk
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*11iTXt__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg$> [1]
>> with an unregistered keyword "XML:com.adobe.xmp". This makes sense to me.
>> Other keywords represent other known metadata such as copyright, author,
>> title, creation time...
>> Perhaps we could add an official keyword for XMP.
>>
>> However, the datastream placement of "iTXt" and similar chunks has no
>> restriction, according to PNG spec part 5.6, Chunk ordering
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*5ChunkOrdering__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw$>
>> [2]. If the XMP data itself could affect how the image is decoded, we would
>> want to enforce that it appears before the "IDAT" chunk in the datastream.
>> Similarly, perhaps XMP would require a different (un)safe-to-copy property.
>>
>> If we need strict chunk ordering or different chunk properties, we might
>> want XMP to appear in a new chunk.
>>
>>
>>
>> I do not know enough about XMP or the future extra metadata that we might
>> be adding. So I cannot make a judgement call about this. I'll spend some
>> time reading about XMP.
>>
>>
>> Interestingly, PNG spec part 12.10, Text chunk processing
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*12Text-chunk-processing__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw$>
>> [3] suggests scenarios where text chunks might want to appear before or
>> after the "IDAT" chunk.
>>
>> [1] https://www.w3.org/TR/2003/REC-PNG-20031110/#11iTXt
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*11iTXt__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg$>
>> [2] https://www.w3.org/TR/2003/REC-PNG-20031110/#5ChunkOrdering
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*5ChunkOrdering__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw$>
>> [3] https://www.w3.org/TR/2003/REC-PNG-20031110/#12Text-chunk-processing
>> <https://urldefense.com/v3/__https:/www.w3.org/TR/2003/REC-PNG-20031110/*12Text-chunk-processing__;Iw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw$>
>>
>