Re: backwards compatible half-float PNG test app source+examples

>And if someone wants to treat the values as something else – such as HDR
data – they can do so, out of scope to the spec…which is also fine(-ish).

If the PNG spec defines the "sBIT" (Significant bits) chunk, I don't see
why it can't define a chunk which indicates the 16-bit components are
actually losslessly encoded half-floats:

"To simplify decoders, PNG specifies that only certain sample depths may be
used, and further specifies that sample values should be scaled to the full
range of possible values at the sample depth. The sBIT chunk defines the
original number of significant bits (which can be less than or equal to the
sample depth)."

On Mon, Nov 13, 2023 at 10:13 AM Leonard Rosenthol <lrosenth@adobe.com>
wrote:

> If you add the `sRGB` tag, so that it is clearly declared that such is the
> case, then at least you have specified the color appropriately.
>
>
>
> And if someone wants to treat the values as something else – such as HDR
> data – they can do so, out of scope to the spec…which is also fine(-ish).
>
>
>
> Leonard
>
>
>
> *From: *Richard Geldreich <rich@binomial.info>
> *Date: *Monday, November 13, 2023 at 10:10 AM
> *To: *Leonard Rosenthol <lrosenth@adobe.com>
> *Cc: *public-png@w3.org <public-png@w3.org>
> *Subject: *Re: backwards compatible half-float PNG test app
> source+examples
>
> *EXTERNAL: Use caution when clicking on links or opening attachments.*
>
>
>
> > BUT if they are “backwards compatible” (as you discuss), then at least
> for that use case, they still need to be relevant to a color space.  And
> what is that?
>
>
>
> If we take an .EXR image downloaded from the web without "chromaticities",
> "whiteLuminance", or "adoptedNeutral" fields (which are defined as strictly
> optional in the spec itself, and aren't always used) - what colorspace is
> the output half-float PNG?
>
>
>
> The high bytes in the half float PNG (which is what Windows and other
> views will preview and view 16-bit PNG's as) in the lossless encoding I've
> proposed will be in sRGB.
>
>
>
> On Mon, Nov 13, 2023 at 10:00 AM Leonard Rosenthol <lrosenth@adobe.com>
> wrote:
>
> But any pixel value is only useful when considered in a given color space
> – unless, as Chris noted in his message, that you don’t consider these
> values for display and only for “data storage”.  And if that is the case,
> then PNG isn’t the right place for them.   BUT if they are “backwards
> compatible” (as you discuss), then at least for that use case, they still
> need to be relevant to a color space.  And what is that?
>
>
>
> Yes, I agree that EXR is a complex format – been working with that team on
> adding C2PA support, so I get it.
>
>
>
> But just because you have a hammer, doesn’t make everything a nail…
>
>
>
> Leonard
>
>
>
> *From: *Richard Geldreich <rich@binomial.info>
> *Date: *Monday, November 13, 2023 at 9:52 AM
> *To: *Leonard Rosenthol <lrosenth@adobe.com>
> *Cc: *public-png@w3.org <public-png@w3.org>
> *Subject: *Re: backwards compatible half-float PNG test app
> source+examples
>
> *EXTERNAL: Use caution when clicking on links or opening attachments.*
>
>
>
> >First off, HDR doesn’t mean anything without (at least) a CICP table.  So
> are you adding a standard cICP tag as well to your images?
>
> The half float encoding != colorspace. Ward: "Because it [half floats] can
> represent negative primary values along with positive ones, the OpenEXR
> format covers the entire visible gamut and a range of about 10.7 orders of
> magnitude with a relative precision of 0.1%." It can handle any colorspace.
> Also according to Ward, many .EXR images don't actually encode the
> *optional* " chromaticities", "whiteLuminance", and "adoptedNeutral" .EXR
> fields in the file, yet somehow we've survived.
>
> >Second, what is the benefit to your published, but proprietary solution
> over an open specification such as OpenEXR?
>
> The Windows OS will not preview or view .EXR files, but it will preview
> and view half-float encoded PNG files if saved correctly in a lossless
> manner which we've already solved.
>
>
>
> "TinyEXR" is 10,000 lines of code, not counting the miniz library for
> Deflate support (which I wrote BTW). It can only load ~70% of the EXR's in
> the wild, so as a reliable interchange format it's terrible.OpenEXR itself
> is too much code and too bloated (dozens of source files).
>
>
>
> The half-float encoding is an industry standard (IEEE 754-2008), and is
> widely supported in GPU hardware and in GPU shader languages:
> https://en.wikipedia.org/wiki/Half-precision_floating-point_format
>
>
>
>
>
> On Mon, Nov 13, 2023 at 9:33 AM Leonard Rosenthol <lrosenth@adobe.com>
> wrote:
>
> Richard – I don’t understand.
>
>
>
> First off, HDR doesn’t mean anything without (at least) a CICP table.  So
> are you adding a standard cICP tag as well to your images?  If you are
> doing that, then at least you’re creating something that can be used in
> other workflows.  This is especially important as you think about how these
> images will be composited with other HDR content, but in other formats…for
> example, consider a web page with both one of your PNGs and a JPEG (using
> the upcoming standard)…
>
>
>
> Second, what is the benefit to your published, but proprietary solution
> over an open specification such as OpenEXR?  Just that it happens to work
> in PNG instead of some other format?  From a wider industry perspective, I
> don’t see the benefit and instead lots of downsides to users worldwide who
> will now be confused why their images don’t work as expected.
>
>
>
> I’m certainly in favor of moving HDR forward – but IMO, we need to do it
> consistently as an industry and not a series of “one offs”….
>
>
>
> Leonard
>
>
>
> *From: *Richard Geldreich <rich@binomial.info>
> *Date: *Monday, November 13, 2023 at 9:10 AM
> *To: *Leonard Rosenthol <lrosenth@adobe.com>
> *Cc: *public-png@w3.org <public-png@w3.org>
> *Subject: *Re: backwards compatible half-float PNG test app
> source+examples
>
> *EXTERNAL: Use caution when clicking on links or opening attachments.*
>
>
>
> > Before introducing another PNG-specific approach, we should see where
> the industry is going and see if we can align with them first (which I
> believe we can/should).
>
>
>
> We need 16-bit/component half float encoding support (not "color" - that's
> a different problem) right now for our HDR codec work, so I'm going to open
> source this couple hundred line HDR PNG solution with a small easy to
> understand readme doc this week, using a ZLIB+Public Domain license. I'll
> then work with PNG library authors to get it supported into their libs and
> modify existing open source viewers to load HDR PNG's.
>
>
>
> We're already using half-float HDR PNG images for testing and development
> in our codec work here. It's backwards compatible with all existing readers
> and OS's, unlike .EXR, so it's a no brainer for us.
>
>
>
> Best regards,
>
> -Rich
>
>
>
> On Mon, Nov 13, 2023 at 8:35 AM Leonard Rosenthol <lrosenth@adobe.com>
> wrote:
>
> Richard – very interesting work, thanks for sharing!
>
>
>
> However, it isn’t in line with where the rest of the industry is going
> with respect to HDR imaging.  Currently ISO TC 42, TC 171, TC 130, JTC 1/SC
> 29 and the ICC are jointly working on a set of proposals for how to encode
> various aspects of HDR data – including gain maps – into common formats and
> technologies (including PNG).  The groups will be meeting in Tokyo in
> February for a full day working session to continue to move this work
> forward.  I believe that Chris Lilley will be there (perhaps remotely?) as
> well as myself.
>
>
>
> We already have some complex logic in PNG with respect to color and HDR
> (cICP vs. iCCP vs. gAMA vs sRGB vs ….).  Before introducing another
> PNG-specific approach, we should see where the industry is going and see if
> we can align with them first (which I believe we can/should).
>
>
>
> Leonard
>
>
>
> *From: *Richard Geldreich <rich@binomial.info>
> *Date: *Friday, November 10, 2023 at 11:05 PM
> *To: *public-png@w3.org <public-png@w3.org>
> *Subject: *Re: backwards compatible half-float PNG test app
> source+examples
>
> *EXTERNAL: Use caution when clicking on links or opening attachments.*
>
>
>
> Here's how the half float pixel values are losslessly recovered from
> 16-bit/component HDR PNG images in the "hdrpng" example code. This is the
> entire decoding procedure.
>
>
>
> This code extracts the low and high bytes from the stored PNG's 16-bpp
> pixels, remaps the high byte through the lookup table read from the "hdRa"
> ancillary chunk (below), shifts right the 16-bit value from [0,15] bits
> (this is typically 0, but for darker images it may be a small # of bits),
> and then rotates the sign bit back into the MSB from the LSB of the 16-bit
> value. This standard half float value can then be processed in an HDR
> pipeline.
>
>
>
> The lossless encoder that takes half-floats and outputs 16-bit PNG values
> is more complex, but not by much. It has to determine the shift amount by
> examining all the half floats in the image and a high byte histogram, and
> then compute a 256 entry lookup table using some sort of tone mapping
> algorithm. The table must be computed in a way that results in no loss.
>
>
>
>
>
> On Fri, Nov 10, 2023 at 4:48 AM Richard Geldreich <rich@binomial.info>
> wrote:
>
> If you want to see these example HDR .PNG's, unpacked to HDR .EXR files,
> using an in-browser HDR viewer, I've unpacked them (using the example
> hdrpng tool) to github here:
>
> https://github.com/richgel999/png16/tree/main/bin/unpacked
>
>
>
> The EXR viewer app (it's pretty good - I use it for testing on SDR
> monitors):
>
> https://viewer.openhdr.org/
>
>
>
>
>
> On Fri, Nov 10, 2023 at 4:34 AM Richard Geldreich <rich@binomial.info>
> wrote:
>
> I've found a lossless and trivally invertible transform that takes
> half-float HDR values (typically read from .HDR or .EXR images) and packs
> them to 16-bit unsigned pixels that are completely compatible with existing
> non-HDR aware PNG software. It uses a simple invertible and lossless global
> tone mapping operator that operates directly on the half-float values. Old
> readers view and see these PNG's as 16-bit PNG's (so usually 48bpp per
> pixel for RGB).
>
>
>
> The high bytes are tone mapped so the 16bpp images appear passable to
> existing readers. New readers can parse a small ~257 byte ancillary chunk
> which contains a byte remapping table used to remap the high bytes of the
> 16-bit components in the PNG image back to half-floats. (It's a little bit
> more complex than this to deal with signed floats and very low values, but
> that's the gist of it.) It's lossless for all valid half-float values
> (normals, denormals, signed). I filter out any NaN's/Inf"s in this test.
>
>
>
> You can see a bunch of example 48-bpp PNG's packed from .EXR images in
> this way here:
>
> https://github.com/richgel999/png16/tree/main/bin
>
>
>
> The C++ source to the example "hdrpng" tool is here. The example app
> excluding image reading/writing is only ~450 lines of code. (It currently
> compiles with VS 2022 under Windows - I'll add a Linux cmake file next.)
>
> https://github.com/richgel999/png16/
>
>
>
> It uses the popular open source lodepng library, unmodified, to write and
> read 16-bit PNG files and manipulate the ancillary "hdRa" chunk.
> hdrpng supports packing and tone mapping .EXR images to .PNG, unpacking HDR
> .PNG to .EXR, and an .EXR file comparison mode to verify that the half
> float values can be 100% recovered from the PNG file with no loss. The tone
> mapping in this example is automatic.
>
>
>
> I've tested the resulting 48bpp PNG files with pngcheck, Windows Explorer,
> Chrome, Paint Shop Pro, several tools, and the Windows Photo viewer app. So
> far, so good - they all look fine.
>
>
>
> Here are a couple example 48bpp PNG files. (Not sure what gmail will do to
> them, but this is what they look like.) These are how existing PNG readers
> view these HDR files. The half-float data is 100% preserved in these .PNG
> files, so HDR capable viewers are able to retrieve the original half-float
> pixels and do their own tone mapping or HDR processing.
>
>
>
>
>
> These PNG's validate successfully using pngcheck, because they are
> completely standard PNG files that any reader can load:
>
>
>
>