Re: the Khronos "glTF" ISO/IEC standard, PNG's, and half float support

I know you asked Rich. I would like to add a thought here, too.

If the PNG still loads reasonably well in non-half-aware decoders, the
spirit and intention of that rule might still be respected.



Additionally, we might want to revisit the rules (albeit, very cautiously).
We redefined iCCP to accept ICC v4, which is a breaking change. But all
decoders we know of already used ICC v4 or none at all. So although it was
a breaking change, we didn't break anything we know of. It would have been
a shame if a "no breaking changes" rule prevented v4 in the spec as it was
already being used.




On Tue, Nov 14, 2023, 10:46 AM Leonard Rosenthol <lrosenth@adobe.com> wrote:

> Richard – thanks for the context of your work with respect to glTF.
> Certainly useful to understand.
>
>
>
> Back to your proposed chunks.
>
>
>
> Using a `haLf` chunk (details TBD) to identify that the 16 bit component
> values should be treated as half floats, seems like a perfectly reasonable
> use for PNG.  As you have noted, it is fully backwards compatible with the
> existing ecosystem and (IMO) aligned with Annex B (Guidelines for private
> chunks, https://www.w3.org/TR/png-3/#newchunks) of PNG.
>
>
>
> The second one you mentioned, however, seems to be (based on the
> description below) to break rule #1 in Annex B:
>
>    1. Do not define new chunks that redefine the meaning of existing
>    chunks or change the interpretation of an existing standardized chunk,
>    e.g., do not add a new chunk to say that RGB and alpha values actually mean
>    CMYK.
>
> If I understand your description below, it is doing exactly what it being
> recommended against.  Would you agree?
>
>
>
> Leonard
>
>
>
> *From: *Richard Geldreich <rich@binomial.info>
> *Date: *Tuesday, November 14, 2023 at 9:20 AM
> *To: *Portable Network Graphics (PNG) Working Group <public-png@w3.org>
> *Cc: *NTREVETT@NVIDIA.COM <NTREVETT@NVIDIA.COM>
> *Subject: *the Khronos "glTF" ISO/IEC standard, PNG's, and half float
> support
>
> *EXTERNAL: Use caution when clicking on links or opening attachments.*
>
>
>
> Note I have CC'd this to Neil Trevett, the Khronos Group President and VP
> of Developer Echosystems at NVidia. NVidia is also an official and current
> W3C Member.
>
>
>
> The Khronos Group, of which Binomial LLC has been a member for several
> years, has created the glTF API-neutral and royalty-free runtime asset
> delivery format. glTF bridges the gap between 3D content creation tools and
> modern graphics applications by providing an efficient, extensible,
> interoperable format for the transmission and loading of 3D content. It's
> also an ISO/IEC 12113:2022 International Standard.
>
>
>
> *Importantly, the glTF specification explicitly refers to the PNG format
> standard by name.* PNG textures are very commonly utilized in glTF assets
> for the delivery of texture signals for 3D content:
>
> https://en.wikipedia.org/wiki/GlTF
>
>
> Some glTF milestones:
> - At SIGGRAPH 2016, Oculus (Meta) announced their adoption of glTF citing
> the similarities to their ovrscene format.
> - In October 2016, Microsoft joined the 3D Formats working group at
> Khronos to collaborate on glTF.
> - In February 2020, the Smithsonian Institution launched their Open Access
> Initiative, releasing approximately 2.8 million 2D images and 3D models
> into the public domain, delivering the 3D models in glTF
> - In July 2022, glTF 2.0 was released as an ISO/IEC 12113:2022
> International Standard.
>
>
>
> Binomial has been a member of the glTF Working Group for several years. We
> have closely worked with Khronos to standardize our Basis Universal GPU
> texture interchange format for usage by glTF, currently for LDR/SDR
> signals. The universal GPU texture codecs for 2D HDR half float
> texture/image signals are currently in production and coming next.
>
> I*mportantly, the glTF spec defines several texture image file formats*.
> A glTF image under section "2.2.3. Technical Terminology" is defined by the
> spec as *"A two dimensional array of pixels encoded as a standardized
> bitstream, such as PNG"*:
>
> https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#reference-texture
>
>
>
> Also see "2.4. glTF Basics", "2.2.4.1. External Specifications", "2.2.4.2.
> Media Type Registrations", etc. *where PNG is explicitly mentioned*. It's
> a common file format utilized by glTF users for the lossless transmission
> and delivery of texture signal data.
>
>
>
> Unfortunately, *as the .PNG specification doesn't yet support half-float
> (IEEE 754-2008 ) HDR texture data (even though the half float format is now
> ubiquitous in computer graphics)*, we're going to *extend the PNG file
> format using a couple private chunks*:
>
> 1. One chunk (tentatively named *"haLf"*) will declare to decoders that
> the16-bit FP16/HALF data is *losslessly and trivially
> inevitably remapped, or tone mapped, inside the PNG file* as standard
> 16-bit sRGB texture data. This will work with L (Greyscale), LA
> (Greyscale-Alpha), RGB, or RGBA textures. Currently the interpretation and
> encoding of the alpha channel data isn't impacted, only L or RGB, but this
> may change to also allow half float alpha data. Existing PNG software will
> read and view this half float data as 16-bit sRGB image/texture data, in a
> backwards compatible manner. All encodable 16-bit half values are supported
> by this chunk: signed, NaN's, InF's, denormals, or normal half float values.
>
> 2. *Another (in progress) optional private ancillary chunk will declare
> which colorspace the half texture data is in*. This chunk will allow a
> lossless conversion from Radiance RGBE HDR/PIC format or .EXR's (optional)
> colorspace data. An optional 2nd colorspace is needed because the
> tonemapped texture data exists in potentially two colorspaces
> simultaneously: sRGB 16-bit for backwards compatibility, and another
> colorspace with a half float encoding.
>
>
>
> See the PNG specification under section 12.10.1 "Use of private chunks":
> "Encoders MAY use private chunks to carry information that need not be
> understood by other applications." As the texture data is delivered as
> 16-bit sRGB, other existing applications don't need to know it's actually
> half float texture data.
>
> Additionally see 5.7.3 Defining private chunks: "An ancillary chunk type,
> not a critical chunk type, SHOULD be used for all private chunks that store
> information that is not absolutely essential to view the image." These
> chunks are not essential to view the image, as the texture data has been
> remapped or losslessly and inevitably tone mapped in a backwards compatible
> manner. Windows, Chrome, Firefox, and existing PNG image editors all
> preview, display and load the 16-bit image data in the usual way, as sRGB
> content.
>
>
>
> I'll be writing the half float chunk definition specs and PNG ecosystem
> software that supports these chunks over the upcoming months, as part of
> our HDR Universal Texture work. We and our customers also use the .PNG file
> format to feed LDR/SDR texture signal data into our Universal GPU texture
> codec, and once this addition is made they'll also be able to input into
> our codec half float texture data. Once we roll this out glTF standard
> users will be able to feed in losslessly compressed HDR (via half float
> PNG) and lossy HDR texture/image signal data (via Basis Universal HDR,
> which will support half float PNG) into glTF, optionally in any colorspace.
>
>
>
> Best regards,
>
> -Rich
>
>
>
>
>