- From: Richard Geldreich <rich@binomial.info>
- Date: Mon, 13 Nov 2023 10:34:32 -0500
- To: Leonard Rosenthol <lrosenth@adobe.com>
- Cc: "public-png@w3.org" <public-png@w3.org>
- Message-ID: <CAMJqMUQGjta8Lca7F4cZubnNSUNBC-M0obkKzQZaaK0VFxUz+Q@mail.gmail.com>
Note the proposed half float encoding exists in *two colorspaces (or encodings) simultaneously *because it's been tone mapped: 16-bits sRGB and another space with a half float encoding, which may be undefined in the file. Because it's in 16-bit sRGB it's backwards compatible with virtually all existing viewers/readers. That's a very powerful property. On Mon, Nov 13, 2023 at 10:25 AM Richard Geldreich <rich@binomial.info> wrote: > It's interesting that the sBIT chunk describes a process quite similar to > what we're doing to the integer bits of half-floats, to get the high bytes > in a form that resembles sRGB pixels. The "sample values should be scaled > to the full range of possible values" - which is exactly what the proposed > half float encoding does to the high bytes after the sign bit is rotated > left into the LSB. > > Half-floats are a IEEE spec, and very popular on GPU's, so I don't see why > PNG cannot define a chunk and a spec for how the standardized half floats > are mapped. > > > On Mon, Nov 13, 2023 at 10:18 AM Richard Geldreich <rich@binomial.info> > wrote: > >> >If you add the `sRGB` tag, so that it is clearly declared that such is >> the case, then at least you have specified the color appropriately. >> >> Not everyone specifies the color appropriately. In practice, PNG is not >> just a format used for *only* viewing files on a monitor. It's also an >> interchange format used between programs. In common cases the colorspace is >> implied or communicated external to .PNG. >> >> Yet users are still able to preview and view their files using common >> OS's, browsers. It's not ideal but the software/OS/tool ecosystem works >> this way in practice. >> >> On Mon, Nov 13, 2023 at 10:13 AM Leonard Rosenthol <lrosenth@adobe.com> >> wrote: >> >>> If you add the `sRGB` tag, so that it is clearly declared that such is >>> the case, then at least you have specified the color appropriately. >>> >>> >>> >>> And if someone wants to treat the values as something else – such as HDR >>> data – they can do so, out of scope to the spec…which is also fine(-ish). >>> >>> >>> >>> Leonard >>> >>> >>> >>> *From: *Richard Geldreich <rich@binomial.info> >>> *Date: *Monday, November 13, 2023 at 10:10 AM >>> *To: *Leonard Rosenthol <lrosenth@adobe.com> >>> *Cc: *public-png@w3.org <public-png@w3.org> >>> *Subject: *Re: backwards compatible half-float PNG test app >>> source+examples >>> >>> *EXTERNAL: Use caution when clicking on links or opening attachments.* >>> >>> >>> >>> > BUT if they are “backwards compatible” (as you discuss), then at least >>> for that use case, they still need to be relevant to a color space. And >>> what is that? >>> >>> >>> >>> If we take an .EXR image downloaded from the web without >>> "chromaticities", "whiteLuminance", or "adoptedNeutral" fields (which are >>> defined as strictly optional in the spec itself, and aren't always used) - >>> what colorspace is the output half-float PNG? >>> >>> >>> >>> The high bytes in the half float PNG (which is what Windows and other >>> views will preview and view 16-bit PNG's as) in the lossless encoding I've >>> proposed will be in sRGB. >>> >>> >>> >>> On Mon, Nov 13, 2023 at 10:00 AM Leonard Rosenthol <lrosenth@adobe.com> >>> wrote: >>> >>> But any pixel value is only useful when considered in a given color >>> space – unless, as Chris noted in his message, that you don’t consider >>> these values for display and only for “data storage”. And if that is the >>> case, then PNG isn’t the right place for them. BUT if they are “backwards >>> compatible” (as you discuss), then at least for that use case, they still >>> need to be relevant to a color space. And what is that? >>> >>> >>> >>> Yes, I agree that EXR is a complex format – been working with that team >>> on adding C2PA support, so I get it. >>> >>> >>> >>> But just because you have a hammer, doesn’t make everything a nail… >>> >>> >>> >>> Leonard >>> >>> >>> >>> *From: *Richard Geldreich <rich@binomial.info> >>> *Date: *Monday, November 13, 2023 at 9:52 AM >>> *To: *Leonard Rosenthol <lrosenth@adobe.com> >>> *Cc: *public-png@w3.org <public-png@w3.org> >>> *Subject: *Re: backwards compatible half-float PNG test app >>> source+examples >>> >>> *EXTERNAL: Use caution when clicking on links or opening attachments.* >>> >>> >>> >>> >First off, HDR doesn’t mean anything without (at least) a CICP table. >>> So are you adding a standard cICP tag as well to your images? >>> >>> The half float encoding != colorspace. Ward: "Because it [half floats] >>> can represent negative primary values along with positive ones, the OpenEXR >>> format covers the entire visible gamut and a range of about 10.7 orders of >>> magnitude with a relative precision of 0.1%." It can handle any colorspace. >>> Also according to Ward, many .EXR images don't actually encode the >>> *optional* " chromaticities", "whiteLuminance", and "adoptedNeutral" >>> .EXR fields in the file, yet somehow we've survived. >>> >>> >Second, what is the benefit to your published, but proprietary solution >>> over an open specification such as OpenEXR? >>> >>> The Windows OS will not preview or view .EXR files, but it will preview >>> and view half-float encoded PNG files if saved correctly in a lossless >>> manner which we've already solved. >>> >>> >>> >>> "TinyEXR" is 10,000 lines of code, not counting the miniz library for >>> Deflate support (which I wrote BTW). It can only load ~70% of the EXR's in >>> the wild, so as a reliable interchange format it's terrible.OpenEXR itself >>> is too much code and too bloated (dozens of source files). >>> >>> >>> >>> The half-float encoding is an industry standard (IEEE 754-2008), and is >>> widely supported in GPU hardware and in GPU shader languages: >>> https://en.wikipedia.org/wiki/Half-precision_floating-point_format >>> >>> >>> >>> >>> >>> On Mon, Nov 13, 2023 at 9:33 AM Leonard Rosenthol <lrosenth@adobe.com> >>> wrote: >>> >>> Richard – I don’t understand. >>> >>> >>> >>> First off, HDR doesn’t mean anything without (at least) a CICP table. >>> So are you adding a standard cICP tag as well to your images? If you are >>> doing that, then at least you’re creating something that can be used in >>> other workflows. This is especially important as you think about how these >>> images will be composited with other HDR content, but in other formats…for >>> example, consider a web page with both one of your PNGs and a JPEG (using >>> the upcoming standard)… >>> >>> >>> >>> Second, what is the benefit to your published, but proprietary solution >>> over an open specification such as OpenEXR? Just that it happens to work >>> in PNG instead of some other format? From a wider industry perspective, I >>> don’t see the benefit and instead lots of downsides to users worldwide who >>> will now be confused why their images don’t work as expected. >>> >>> >>> >>> I’m certainly in favor of moving HDR forward – but IMO, we need to do it >>> consistently as an industry and not a series of “one offs”…. >>> >>> >>> >>> Leonard >>> >>> >>> >>> *From: *Richard Geldreich <rich@binomial.info> >>> *Date: *Monday, November 13, 2023 at 9:10 AM >>> *To: *Leonard Rosenthol <lrosenth@adobe.com> >>> *Cc: *public-png@w3.org <public-png@w3.org> >>> *Subject: *Re: backwards compatible half-float PNG test app >>> source+examples >>> >>> *EXTERNAL: Use caution when clicking on links or opening attachments.* >>> >>> >>> >>> > Before introducing another PNG-specific approach, we should see where >>> the industry is going and see if we can align with them first (which I >>> believe we can/should). >>> >>> >>> >>> We need 16-bit/component half float encoding support (not "color" - >>> that's a different problem) right now for our HDR codec work, so I'm going >>> to open source this couple hundred line HDR PNG solution with a small easy >>> to understand readme doc this week, using a ZLIB+Public Domain license. >>> I'll then work with PNG library authors to get it supported into their libs >>> and modify existing open source viewers to load HDR PNG's. >>> >>> >>> >>> We're already using half-float HDR PNG images for testing and >>> development in our codec work here. It's backwards compatible with all >>> existing readers and OS's, unlike .EXR, so it's a no brainer for us. >>> >>> >>> >>> Best regards, >>> >>> -Rich >>> >>> >>> >>> On Mon, Nov 13, 2023 at 8:35 AM Leonard Rosenthol <lrosenth@adobe.com> >>> wrote: >>> >>> Richard – very interesting work, thanks for sharing! >>> >>> >>> >>> However, it isn’t in line with where the rest of the industry is going >>> with respect to HDR imaging. Currently ISO TC 42, TC 171, TC 130, JTC 1/SC >>> 29 and the ICC are jointly working on a set of proposals for how to encode >>> various aspects of HDR data – including gain maps – into common formats and >>> technologies (including PNG). The groups will be meeting in Tokyo in >>> February for a full day working session to continue to move this work >>> forward. I believe that Chris Lilley will be there (perhaps remotely?) as >>> well as myself. >>> >>> >>> >>> We already have some complex logic in PNG with respect to color and HDR >>> (cICP vs. iCCP vs. gAMA vs sRGB vs ….). Before introducing another >>> PNG-specific approach, we should see where the industry is going and see if >>> we can align with them first (which I believe we can/should). >>> >>> >>> >>> Leonard >>> >>> >>> >>> *From: *Richard Geldreich <rich@binomial.info> >>> *Date: *Friday, November 10, 2023 at 11:05 PM >>> *To: *public-png@w3.org <public-png@w3.org> >>> *Subject: *Re: backwards compatible half-float PNG test app >>> source+examples >>> >>> *EXTERNAL: Use caution when clicking on links or opening attachments.* >>> >>> >>> >>> Here's how the half float pixel values are losslessly recovered from >>> 16-bit/component HDR PNG images in the "hdrpng" example code. This is the >>> entire decoding procedure. >>> >>> >>> >>> This code extracts the low and high bytes from the stored PNG's 16-bpp >>> pixels, remaps the high byte through the lookup table read from the "hdRa" >>> ancillary chunk (below), shifts right the 16-bit value from [0,15] bits >>> (this is typically 0, but for darker images it may be a small # of bits), >>> and then rotates the sign bit back into the MSB from the LSB of the 16-bit >>> value. This standard half float value can then be processed in an HDR >>> pipeline. >>> >>> >>> >>> The lossless encoder that takes half-floats and outputs 16-bit PNG >>> values is more complex, but not by much. It has to determine the shift >>> amount by examining all the half floats in the image and a high byte >>> histogram, and then compute a 256 entry lookup table using some sort of >>> tone mapping algorithm. The table must be computed in a way that results in >>> no loss. >>> >>> >>> >>> >>> >>> On Fri, Nov 10, 2023 at 4:48 AM Richard Geldreich <rich@binomial.info> >>> wrote: >>> >>> If you want to see these example HDR .PNG's, unpacked to HDR .EXR files, >>> using an in-browser HDR viewer, I've unpacked them (using the example >>> hdrpng tool) to github here: >>> >>> https://github.com/richgel999/png16/tree/main/bin/unpacked >>> >>> >>> >>> The EXR viewer app (it's pretty good - I use it for testing on SDR >>> monitors): >>> >>> https://viewer.openhdr.org/ >>> >>> >>> >>> >>> >>> On Fri, Nov 10, 2023 at 4:34 AM Richard Geldreich <rich@binomial.info> >>> wrote: >>> >>> I've found a lossless and trivally invertible transform that takes >>> half-float HDR values (typically read from .HDR or .EXR images) and packs >>> them to 16-bit unsigned pixels that are completely compatible with existing >>> non-HDR aware PNG software. It uses a simple invertible and lossless global >>> tone mapping operator that operates directly on the half-float values. Old >>> readers view and see these PNG's as 16-bit PNG's (so usually 48bpp per >>> pixel for RGB). >>> >>> >>> >>> The high bytes are tone mapped so the 16bpp images appear passable to >>> existing readers. New readers can parse a small ~257 byte ancillary chunk >>> which contains a byte remapping table used to remap the high bytes of the >>> 16-bit components in the PNG image back to half-floats. (It's a little bit >>> more complex than this to deal with signed floats and very low values, but >>> that's the gist of it.) It's lossless for all valid half-float values >>> (normals, denormals, signed). I filter out any NaN's/Inf"s in this test. >>> >>> >>> >>> You can see a bunch of example 48-bpp PNG's packed from .EXR images in >>> this way here: >>> >>> https://github.com/richgel999/png16/tree/main/bin >>> >>> >>> >>> The C++ source to the example "hdrpng" tool is here. The example app >>> excluding image reading/writing is only ~450 lines of code. (It currently >>> compiles with VS 2022 under Windows - I'll add a Linux cmake file next.) >>> >>> https://github.com/richgel999/png16/ >>> >>> >>> >>> It uses the popular open source lodepng library, unmodified, to write >>> and read 16-bit PNG files and manipulate the ancillary "hdRa" chunk. >>> hdrpng supports packing and tone mapping .EXR images to .PNG, unpacking HDR >>> .PNG to .EXR, and an .EXR file comparison mode to verify that the half >>> float values can be 100% recovered from the PNG file with no loss. The tone >>> mapping in this example is automatic. >>> >>> >>> >>> I've tested the resulting 48bpp PNG files with pngcheck, Windows >>> Explorer, Chrome, Paint Shop Pro, several tools, and the Windows Photo >>> viewer app. So far, so good - they all look fine. >>> >>> >>> >>> Here are a couple example 48bpp PNG files. (Not sure what gmail will do >>> to them, but this is what they look like.) These are how existing PNG >>> readers view these HDR files. The half-float data is 100% preserved in >>> these .PNG files, so HDR capable viewers are able to retrieve the original >>> half-float pixels and do their own tone mapping or HDR processing. >>> >>> >>> >>> >>> >>> These PNG's validate successfully using pngcheck, because they are >>> completely standard PNG files that any reader can load: >>> >>> >>> >>>
Attachments
- image/png attachment: Kapaa.png
- image/png attachment: memorial.png
- image/png attachment: image.png
- image/png attachment: 04-image.png
- image/png attachment: 05-image.png
Received on Monday, 13 November 2023 15:35:06 UTC