Re: backwards compatible half-float PNG test app source+examples

But if the data isn’t a color image – then it doesn’t (IMO) belong in PNG.   That’s why, for example, the ISO JPEG committee is doing their work on “event-based vision” (ala motion vectors) as a separate work item – JPEG XE, https://jpeg.org/jpegxe/index.html


Leonard

From: Chris Blume (ProgramMax) <programmax@gmail.com>
Date: Monday, November 13, 2023 at 9:53 AM
To: Leonard Rosenthol <lrosenth@adobe.com>
Cc: Richard Geldreich <rich@binomial.info>, public-png@w3.org <public-png@w3.org>
Subject: Re: backwards compatible half-float PNG test app source+examples

EXTERNAL: Use caution when clicking on links or opening attachments.


My understanding is Rich is looking at data storage format for image processing, rather than final display.

For example, a 12-bit int might be plenty for displaying a HDR image on the web. But it is not enough for a camera who's captured image will later be processed.

This probably effects things like CICP markings, right?

And if the data isn't actually a color image (infrared, motion vectors) then CICP definitely doesn't apply.

On Mon, Nov 13, 2023, 9:34 AM Leonard Rosenthol <lrosenth@adobe.com<mailto:lrosenth@adobe.com>> wrote:
Richard – I don’t understand.

First off, HDR doesn’t mean anything without (at least) a CICP table.  So are you adding a standard cICP tag as well to your images?  If you are doing that, then at least you’re creating something that can be used in other workflows.  This is especially important as you think about how these images will be composited with other HDR content, but in other formats…for example, consider a web page with both one of your PNGs and a JPEG (using the upcoming standard)…

Second, what is the benefit to your published, but proprietary solution over an open specification such as OpenEXR?  Just that it happens to work in PNG instead of some other format?  From a wider industry perspective, I don’t see the benefit and instead lots of downsides to users worldwide who will now be confused why their images don’t work as expected.

I’m certainly in favor of moving HDR forward – but IMO, we need to do it consistently as an industry and not a series of “one offs”….

Leonard

From: Richard Geldreich <rich@binomial.info<mailto:rich@binomial.info>>
Date: Monday, November 13, 2023 at 9:10 AM
To: Leonard Rosenthol <lrosenth@adobe.com<mailto:lrosenth@adobe.com>>
Cc: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: Re: backwards compatible half-float PNG test app source+examples

EXTERNAL: Use caution when clicking on links or opening attachments.


> Before introducing another PNG-specific approach, we should see where the industry is going and see if we can align with them first (which I believe we can/should).

We need 16-bit/component half float encoding support (not "color" - that's a different problem) right now for our HDR codec work, so I'm going to open source this couple hundred line HDR PNG solution with a small easy to understand readme doc this week, using a ZLIB+Public Domain license. I'll then work with PNG library authors to get it supported into their libs and modify existing open source viewers to load HDR PNG's.

We're already using half-float HDR PNG images for testing and development in our codec work here. It's backwards compatible with all existing readers and OS's, unlike .EXR, so it's a no brainer for us.

Best regards,
-Rich

On Mon, Nov 13, 2023 at 8:35 AM Leonard Rosenthol <lrosenth@adobe.com<mailto:lrosenth@adobe.com>> wrote:
Richard – very interesting work, thanks for sharing!

However, it isn’t in line with where the rest of the industry is going with respect to HDR imaging.  Currently ISO TC 42, TC 171, TC 130, JTC 1/SC 29 and the ICC are jointly working on a set of proposals for how to encode various aspects of HDR data – including gain maps – into common formats and technologies (including PNG).  The groups will be meeting in Tokyo in February for a full day working session to continue to move this work forward.  I believe that Chris Lilley will be there (perhaps remotely?) as well as myself.

We already have some complex logic in PNG with respect to color and HDR (cICP vs. iCCP vs. gAMA vs sRGB vs ….).  Before introducing another PNG-specific approach, we should see where the industry is going and see if we can align with them first (which I believe we can/should).

Leonard

From: Richard Geldreich <rich@binomial.info<mailto:rich@binomial.info>>
Date: Friday, November 10, 2023 at 11:05 PM
To: public-png@w3.org<mailto:public-png@w3.org> <public-png@w3.org<mailto:public-png@w3.org>>
Subject: Re: backwards compatible half-float PNG test app source+examples

EXTERNAL: Use caution when clicking on links or opening attachments.


Here's how the half float pixel values are losslessly recovered from 16-bit/component HDR PNG images in the "hdrpng" example code. This is the entire decoding procedure.

This code extracts the low and high bytes from the stored PNG's 16-bpp pixels, remaps the high byte through the lookup table read from the "hdRa" ancillary chunk (below), shifts right the 16-bit value from [0,15] bits (this is typically 0, but for darker images it may be a small # of bits), and then rotates the sign bit back into the MSB from the LSB of the 16-bit value. This standard half float value can then be processed in an HDR pipeline.

The lossless encoder that takes half-floats and outputs 16-bit PNG values is more complex, but not by much. It has to determine the shift amount by examining all the half floats in the image and a high byte histogram, and then compute a 256 entry lookup table using some sort of tone mapping algorithm. The table must be computed in a way that results in no loss.
[cid:ii_18bc8f9d05fcb971f164]

[cid:ii_18bc8f9d05fcb971f165]

On Fri, Nov 10, 2023 at 4:48 AM Richard Geldreich <rich@binomial.info<mailto:rich@binomial.info>> wrote:
If you want to see these example HDR .PNG's, unpacked to HDR .EXR files, using an in-browser HDR viewer, I've unpacked them (using the example hdrpng tool) to github here:
https://github.com/richgel999/png16/tree/main/bin/unpacked

The EXR viewer app (it's pretty good - I use it for testing on SDR monitors):
https://viewer.openhdr.org/



On Fri, Nov 10, 2023 at 4:34 AM Richard Geldreich <rich@binomial.info<mailto:rich@binomial.info>> wrote:
I've found a lossless and trivally invertible transform that takes half-float HDR values (typically read from .HDR or .EXR images) and packs them to 16-bit unsigned pixels that are completely compatible with existing non-HDR aware PNG software. It uses a simple invertible and lossless global tone mapping operator that operates directly on the half-float values. Old readers view and see these PNG's as 16-bit PNG's (so usually 48bpp per pixel for RGB).

The high bytes are tone mapped so the 16bpp images appear passable to existing readers. New readers can parse a small ~257 byte ancillary chunk which contains a byte remapping table used to remap the high bytes of the 16-bit components in the PNG image back to half-floats. (It's a little bit more complex than this to deal with signed floats and very low values, but that's the gist of it.) It's lossless for all valid half-float values (normals, denormals, signed). I filter out any NaN's/Inf"s in this test.

You can see a bunch of example 48-bpp PNG's packed from .EXR images in this way here:
https://github.com/richgel999/png16/tree/main/bin


The C++ source to the example "hdrpng" tool is here. The example app excluding image reading/writing is only ~450 lines of code. (It currently compiles with VS 2022 under Windows - I'll add a Linux cmake file next.)
https://github.com/richgel999/png16/


It uses the popular open source lodepng library, unmodified, to write and read 16-bit PNG files and manipulate the ancillary "hdRa" chunk. hdrpng supports packing and tone mapping .EXR images to .PNG, unpacking HDR .PNG to .EXR, and an .EXR file comparison mode to verify that the half float values can be 100% recovered from the PNG file with no loss. The tone mapping in this example is automatic.

I've tested the resulting 48bpp PNG files with pngcheck, Windows Explorer, Chrome, Paint Shop Pro, several tools, and the Windows Photo viewer app. So far, so good - they all look fine.

Here are a couple example 48bpp PNG files. (Not sure what gmail will do to them, but this is what they look like.) These are how existing PNG readers view these HDR files. The half-float data is 100% preserved in these .PNG files, so HDR capable viewers are able to retrieve the original half-float pixels and do their own tone mapping or HDR processing.

[cid:ii_18bc8f9d05e9c0c09951]
[cid:ii_18bc8f9d05ef2121e172]

These PNG's validate successfully using pngcheck, because they are completely standard PNG files that any reader can load:

[cid:ii_18bc8f9d05fcb971f163]