Fwd: half-float PNG files stored as 16-bit/component images

-------- Forwarded Message --------
Subject:  [Moderator Action (size limit exceeded)] half-float PNG files 
stored as 16-bit/component images
Date:  Mon, 06 Nov 2023 17:27:57 +0000
From:  Richard Geldreich <rich@binomial.info>
To:  public-png@w3.org



Hi all,
*The one core strength of the PNG image format: predictable backwards 
compatibility.* If we introduce a new Colour Type value (where 
#7="16-bit half float RGBA" for example), /existing PNG decoders will 
fail to read half-float PNG files/. Users will then perceive PNG as an 
unreliable format, which is terrible for a standard file format (if not 
terminal). It could take many years for developers to add proper 
half-float support, and some old libs or software will never be updated. 
PNG currently works everywhere and is very reliable.

*Crucially, Jim Blinn said years ago 
<https://stackoverflow.com/questions/75772363/why-does-the-integer-representation-of-a-floating-point-number-offer-a-piecewise>*: 
"/If you only deal with positive numbers, the bit pattern of a floating 
point number, interpreted as an integer, gives a piecewise linear 
approximation to the logarithm function/".

This holds true for positive half floats interpreted as unsigned 16-bit 
values as well. Half floats 
<https://en.wikipedia.org/wiki/Half-precision_floating-point_format> are 
much like 32-bit floats, just with less mantissa/exponent bits. (This is 
how the BC6H/ASTC HDR GPU texture formats work internally, BTW: the half 
float values are manipulated and interpolated as 16-bit unsigned 
integers. BC6H/ASTC encoders work in half-float space, which is an 
approximation of a logarithmic space.)

One true HDR PNG option is to store the half-float pixel values as plain 
16-bit/component PNG Truecolour images (Colour Type=2, Bit Depth=16), 
with a special new ancillary chunk that hints to the reader software 
that it's actually a half-float image and not unsigned. *But crucially, 
what will loaded or displayed images look like when processed by legacy 
PNG software? It turns out the results are surprisingly decent.*

Some popular PNG readers (such as stb_image.h, and mine) load 
16-bit/component images by shifting each component right by 8 bits, to 
return the uppermost 8-bits (so 24 or 32 bits total for RGB/RGBA) to the 
caller. In half float space, this is approximately equivalent to 
floor(log2(value)*scale/256.0). *It's a built-in global tone map 
operator. *The upper 8-bits contain the sign (almost always positive, so 
0), the 5-bit exponent, and the upper 2-bits of the 10-bit mantissa.

Here's an HDR example: *candle-glass.exr*, with a max float component 
value of 413.25, converted to half float. I then extracted the high 
8-bits of these half floats to make plain 24-bit images for 
visualization. Here's what the result looks like (and what a 16-bit half 
float PNG would look like loaded with legacy software):

image.png

After a histogram stretch (using Paint Shop Pro):
image.png

Alternatively, after a quick brightness/contrast adjustment:
image.png

This is a very simple/crude global tone map operator. The user can still 
load the image with old software, they will just have to adjust the 
brightness/contrast, which is probably fine for a true HDR PNG image 
loaded with legacy software.

Another example (MtTamWest.exr):

image.png

After quick brightness/contrast adjustment:

image.png