half-float PNG files stored as 16-bit/component images

Hi all,
*The one core strength of the PNG image format: predictable backwards
compatibility.* If we introduce a new Colour Type value (where #7="16-bit
half float RGBA" for example), *existing PNG decoders will fail to read
half-float PNG files*. Users will then perceive PNG as an unreliable
format, which is terrible for a standard file format (if not terminal). It
could take many years for developers to add proper half-float support, and
some old libs or software will never be updated. PNG currently works
everywhere and is very reliable.

*Crucially, Jim Blinn said years ago
<https://stackoverflow.com/questions/75772363/why-does-the-integer-representation-of-a-floating-point-number-offer-a-piecewise>*
: "*If you only deal with positive numbers, the bit pattern of a floating
point number, interpreted as an integer, gives a piecewise linear
approximation to the logarithm function*".

This holds true for positive half floats interpreted as unsigned 16-bit
values as well. Half floats
<https://en.wikipedia.org/wiki/Half-precision_floating-point_format> are
much like 32-bit floats, just with less mantissa/exponent bits. (This is
how the BC6H/ASTC HDR GPU texture formats work internally, BTW: the half
float values are manipulated and interpolated as 16-bit unsigned integers.
BC6H/ASTC encoders work in half-float space, which is an approximation of a
logarithmic space.)

One true HDR PNG option is to store the half-float pixel values as plain
16-bit/component PNG Truecolour images (Colour Type=2, Bit Depth=16), with
a special new ancillary chunk that hints to the reader software that it's
actually a half-float image and not unsigned. *But crucially, what will
loaded or displayed images look like when processed by legacy PNG software?
It turns out the results are surprisingly decent.*

Some popular PNG readers (such as stb_image.h, and mine) load
16-bit/component images by shifting each component right by 8 bits, to
return the uppermost 8-bits (so 24 or 32 bits total for RGB/RGBA) to the
caller. In half float space, this is approximately equivalent to
floor(log2(value)*scale/256.0). *It's a built-in global tone map operator. *The
upper 8-bits contain the sign (almost always positive, so 0), the 5-bit
exponent, and the upper 2-bits of the 10-bit mantissa.

Here's an HDR example: *candle-glass.exr*, with a max float component value
of 413.25, converted to half float. I then extracted the high 8-bits of
these half floats to make plain 24-bit images for visualization. Here's
what the result looks like (and what a 16-bit half float PNG would look
like loaded with legacy software):

[image: image.png]

After a histogram stretch (using Paint Shop Pro):
[image: image.png]

Alternatively, after a quick brightness/contrast adjustment:
[image: image.png]

This is a very simple/crude global tone map operator. The user can still
load the image with old software, they will just have to adjust the
brightness/contrast, which is probably fine for a true HDR PNG image loaded
with legacy software.

Another example (MtTamWest.exr):

[image: image.png]

After quick brightness/contrast adjustment:

[image: image.png]

Received on Tuesday, 7 November 2023 03:35:47 UTC