Fwd: Re: half-float PNG files stored as 16-bit/component images

-------- Forwarded Message --------
Subject:  [Moderator Action (size limit exceeded)] Re: half-float PNG 
files stored as 16-bit/component images
Date:  Mon, 06 Nov 2023 17:50:45 +0000
From:  Richard Geldreich <rich@binomial.info>
To:  public-png@w3.org



Another option is to store the half-float values as 16-bit unsigned but 
rotated left one bit, so the half-float sign bit (which is typically 0, 
and is stored in the MSB or bit position #15) becomes the LSB (bit 
position #0). This is a lossless operation and could be quickly undone 
by half-float aware PNG readers. However legacy PNG readers would get a 
brighter and less noisy image if they only pay attention to the upper 
8-bits. This gets 5-bits of the exponent and 3 bits for the mantissa (so 
4 bits total because the leading '1' bit isn't explicitly stored). I 
really like this option.

image.png

On Mon, Nov 6, 2023 at 12:27 PM Richard Geldreich <rich@binomial.info> 
wrote:

    Hi all,
    *The one core strength of the PNG image format: predictable
    backwards compatibility.* If we introduce a new Colour Type value
    (where #7="16-bit half float RGBA" for example), /existing PNG
    decoders will fail to read half-float PNG files/. Users will then
    perceive PNG as an unreliable format, which is terrible for a
    standard file format (if not terminal). It could take many years for
    developers to add proper half-float support, and some old libs or
    software will never be updated. PNG currently works everywhere and
    is very reliable.

    *Crucially, Jim Blinn said years ago
    <https://stackoverflow.com/questions/75772363/why-does-the-integer-representation-of-a-floating-point-number-offer-a-piecewise>*:
    "/If you only deal with positive numbers, the bit pattern of a
    floating point number, interpreted as an integer, gives a piecewise
    linear approximation to the logarithm function/".

    This holds true for positive half floats interpreted as unsigned
    16-bit values as well. Half floats
    <https://en.wikipedia.org/wiki/Half-precision_floating-point_format>
    are much like 32-bit floats, just with less mantissa/exponent bits.
    (This is how the BC6H/ASTC HDR GPU texture formats work
    internally, BTW: the half float values are manipulated and
    interpolated as 16-bit unsigned integers. BC6H/ASTC encoders work in
    half-float space, which is an approximation of a logarithmic space.)

    One true HDR PNG option is to store the half-float pixel values as
    plain 16-bit/component PNG Truecolour images (Colour Type=2, Bit
    Depth=16), with a special new ancillary chunk that hints to the
    reader software that it's actually a half-float image and not
    unsigned. *But crucially, what will loaded or displayed images look
    like when processed by legacy PNG software? It turns out the results
    are surprisingly decent.*

    Some popular PNG readers (such as stb_image.h, and mine) load
    16-bit/component images by shifting each component right by 8 bits,
    to return the uppermost 8-bits (so 24 or 32 bits total for RGB/RGBA)
    to the caller. In half float space, this is approximately equivalent
    to floor(log2(value)*scale/256.0). *It's a built-in global tone map
    operator. *The upper 8-bits contain the sign (almost always
    positive, so 0), the 5-bit exponent, and the upper 2-bits of the
    10-bit mantissa.

    Here's an HDR example: *candle-glass.exr*, with a max float
    component value of 413.25, converted to half float. I then extracted
    the high 8-bits of these half floats to make plain 24-bit images for
    visualization. Here's what the result looks like (and what a 16-bit
    half float PNG would look like loaded with legacy software):

    image.png

    After a histogram stretch (using Paint Shop Pro):
    image.png

    Alternatively, after a quick brightness/contrast adjustment:
    image.png

    This is a very simple/crude global tone map operator. The user can
    still load the image with old software, they will just have to
    adjust the brightness/contrast, which is probably fine for a true
    HDR PNG image loaded with legacy software.

    Another example (MtTamWest.exr):

    image.png

    After quick brightness/contrast adjustment:

    image.png