Re: [EXTERNAL] XMP in PNG

I’ll ask.

From: Chris Blume (ProgramMax) <programmax@gmail.com>
Date: Tuesday, November 30, 2021 at 11:46 AM
To: Leonard Rosenthol <lrosenth@adobe.com>
Cc: Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com>, public-png@w3.org <public-png@w3.org>
Subject: Re: [EXTERNAL] XMP in PNG
According to the public list of W3C member organizations<https://urldefense.com/v3/__https:/www.w3.org/Consortium/Member/List__;!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXwR3U6IAg$> [1], Dolby is not a W3C member.
Does anyone here have a contact at Dolby? It would be very helpful if Dolby became a member and a Dolby employee could join the conversation.

[1] https://www.w3.org/Consortium/Member/List<https://urldefense.com/v3/__https:/www.w3.org/Consortium/Member/List__;!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXwR3U6IAg$>

On Tue, Nov 30, 2021 at 11:33 AM Leonard Rosenthol <lrosenth@adobe.com<mailto:lrosenth@adobe.com>> wrote:
I agree. XMP is a bit heavyweight for what you are proposing.  It could be used, but I think we can do something better/more optimized.

Leonard

From: Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com<mailto:Chris.Seeger@nbcuni.com>>
Date: Tuesday, November 30, 2021 at 11:08 AM
To: Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>>
Cc: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: Re: [EXTERNAL] XMP in PNG
I think XMP is OK for static metadata but not dynamic metadata.

There are standards for “serializing” the dynamic metadata that might be much better as they are “time-based” serializations.  Dolby could comment on this if we can bring them on-board.

-Chris

From: Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>>
Date: Tuesday, November 30, 2021 at 11:06 AM
To: Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com<mailto:Chris.Seeger@nbcuni.com>>
Cc: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: Re: [EXTERNAL] XMP in PNG
Oops. I didn't get my correction email out in time. Your email came in right as I was hitting "send".

I understand better now. Thank you.

Do you feel iTXt + XMP is the correct delivery method for this (dynamic) metadata? If so, should the schema be part of the PNG spec?

On Tue, Nov 30, 2021 at 11:02 AM Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>> wrote:
Perhaps correcting myself:
Rather than metadata providing average brightness, it could provide an array of tone mapping options based on the OTHER displayed content's average brightness. And thus it would be a 1-time chunk, not a per-frame chunk.

Or maybe it is a combination of the two.

(A decoder could track average brightness easily enough for still images. Or for images where a full frame update happens. It just tracks it while it decodes. However, since APNG could update just a subregion of the image this is more difficult. Providing per-frame average brightness could be useful.)

On Tue, Nov 30, 2021 at 10:46 AM Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>> wrote:
I believe I partly understand.
I remember working on HDR on Mac and seeing a video play that affected the brightness of more than just the video. If I remember correctly, the entire display's brightness changed as the video's brightness changed.

For example, if a bright flash is displayed and the viewer's pupils close, the rest of the display will have to become more bright to compensate and remain approximately the same color to the viewer.

Is my understanding correct?
(Is this considered part of the viewing environment?)


If I am on the right track, I can understand why extra metadata is useful.
The average brightness changes throughout the animation. And so we might have this metadata before every frame of animation. Or perhaps every other frame or only when a significant brightness change occurs.

Would we want to include this average brightness as part of the standard? If so, we are specifying a standard schema, too.
(Is iTXt + XMP + schema overhead too heavyweight for a simple average brightness?)

On Tue, Nov 30, 2021 at 9:54 AM Seeger, Chris (NBCUniversal) <Chris.Seeger@nbcuni.com<mailto:Chris.Seeger@nbcuni.com>> wrote:
There could be additional metadata for specific HDR transfer functions that describes specific tone mapping or specific “looks”.

In the case of animated PNG’s, I’m anticipating this could also include dynamic metadata for HDR that is used for tone mapping against overlaid video that also includes dynamic metadata (and therefore could change it’s average brightness level which would in turn affect the overlaid graphics).

-Chris

From: Chris Blume (ProgramMax) <programmax@gmail.com<mailto:programmax@gmail.com>>
Date: Tuesday, November 30, 2021 at 7:03 AM
To: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: [EXTERNAL] XMP in PNG
Solicited feedback:
Chris Seeger, are you allowed to tell us what extra metadata you foresee coming around the corner? It would help us plan.
Everyone, feel free to weigh in.


Thoughts:
In our last meeting, we discussed adding XMP to PNG.
In this email thread we can talk about how we would like to accomplish that.

The motivation is from Chris Seeger, who asked if extra metadata could be added to Exif. Apparently, Exif is rigid and will not allow extra metadata. Leonard Rosenthol proposed XMP as an extensible metadata solution.

Here is the latest Exif standard (translated to English): https://www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.cipa.jp*2Fstd*2Fdocuments*2Fe*2FDC-X008-Translation-2019-E.pdf__*3B!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrx6K1JA-w*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395039725*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=lJ7*2Fa9a9dF*2BNYnKhcrpxUUypROOPCDPwNth7bb0hOqY*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXzWjU5TjQ$>
Here is a link to purchase ISO 16684-1, the XMP standard: https://www.iso.org/standard/75163.html<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.iso.org*2Fstandard*2F75163.html__*3B!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSry98HW4Ig*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395049690*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=P4jYmkF6ADl10rEV6GOObxDmSRsej5cdeUALAG7XblI*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXxo7hKY1g$>



During the meeting, Chris Lilley mentioned that XMP in PNG could go inside the "iTXt" chunk<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*11iTXt__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395059643*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=fRjLQG*2FoD1X4sp0Qu9LkWh4RZ1Cb7rUchm7kyfNrAaI*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUlJQ!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXyj2gbvaw$> [1] with an unregistered keyword "XML:com.adobe.xmp". This makes sense to me. Other keywords represent other known metadata such as copyright, author, title, creation time...
Perhaps we could add an official keyword for XMP.

However, the datastream placement of "iTXt" and similar chunks has no restriction, according to PNG spec part 5.6, Chunk ordering<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*5ChunkOrdering__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395069604*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=ze40wIqOi1rLSZIsD23wuOGUnIWgddvMOESfvJq3TzM*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUl!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXxXQGbr_Q$> [2]. If the XMP data itself could affect how the image is decoded, we would want to enforce that it appears before the "IDAT" chunk in the datastream. Similarly, perhaps XMP would require a different (un)safe-to-copy property.

If we need strict chunk ordering or different chunk properties, we might want XMP to appear in a new chunk.



I do not know enough about XMP or the future extra metadata that we might be adding. So I cannot make a judgement call about this. I'll spend some time reading about XMP.


Interestingly, PNG spec part 12.10, Text chunk processing<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*12Text-chunk-processing__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395079558*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=A2CE6URb3FcGvzCnw8gbGebpTK8sGMpDOPSXchG1CZc*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUl!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXwYK_xOGA$> [3] suggests scenarios where text chunks might want to appear before or after the "IDAT" chunk.

[1] https://www.w3.org/TR/2003/REC-PNG-20031110/#11iTXt<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*11iTXt__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxZz-qPMg*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395089525*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=DWeQNUmTiG14H*2FhqNjCeTONlNO2JVCMxh7sWSXvAxhs*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUlJQ!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXyZhpx97w$>
[2] https://www.w3.org/TR/2003/REC-PNG-20031110/#5ChunkOrdering<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*5ChunkOrdering__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrwUNInpEw*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395089525*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=1p7VGs1ShWbjSLmNSNsS*2FRTLbM6zwKSyf3fG9TDbr4k*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUlJQ!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXyjX0_EPg$>
[3] https://www.w3.org/TR/2003/REC-PNG-20031110/#12Text-chunk-processing<https://urldefense.com/v3/__https:/nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.w3.org*2FTR*2F2003*2FREC-PNG-20031110*2F*12Text-chunk-processing__*3BIw!!PIZeeW5wscynRQ!5xo5SyyJmhmS2tKvFfoz9JbeaM9jmWOpYdgZ3BweBNosffNd6hMMzRgPSrxYT81uPw*24&data=04*7C01*7Clrosenth*40adobe.com*7C185e84670ce440b925ba08d9b41bb686*7Cfa7b1b5a7b34438794aed2c178decee1*7C0*7C0*7C637738853395099485*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=LsqpWiQpD0TH7psTfLkFE*2B53IKWTFUwjeiSkrS3kZCc*3D&reserved=0__;JSUlJSUlJSUlJSUqJSUlJSUlJSUlJSUlJSUlJQ!!PIZeeW5wscynRQ!7d5GwbHKIM-7SLUzVkDtphiEWv0p35RUWachHkc7skiISOrw99Rw2crDyXyu0Nen-A$>