Re: [PNG] Meta just released OpenZL -- compression research

Here is my summary of it, given my current understanding:

   - There are several compressors built in for it to choose from. It falls
   back to Zstd.
   - You describe your data layout using Simple Data Description Language
   (SDDL) <https://openzl.org/api/c/graphs/sddl/>
   - You then train a plan using example files and your data layout (so it
   can understand the files). The training is where it decides how to
   rearrange the data and what compressor (compressors?) to use.
   - The plan is stored in the compressed file so the original file can be
   reconstructed. This leaves open the opportunity for better plan development
   later. Maybe a new feature is added to SDDL to indicate data relationships,
   for example. (This might already exist, I'm skimming.)

I might fiddle around with it and see if/how OpenZL relates to PNGs. I
don't think I can create a SDDL file to describe PNGs, because they already
contain compressed data. I could perhaps describe all the rest like the
chunks and their content. I am curious to see if it allows for future,
undefined chunks. Specifically, "now skip over <chunk length> bytes".

That said, there is an example to use OpenZL with BMP files. So perhaps I
can compare OpenZL-compressed BMPs to PNGs and gain some insight.


My big takeaways are:

   - Other than the IDAT chunk and a few other chunks, most PNG chunks are
   uncompressed. Most are also small so compressing them individually (so
   they're skippable when parsing) probably wouldn't provide benefit. I think
   any gains over PNG would be matched (given the skippability constraint)
   just by allowing that compressor within IDAT.
   - It is all basically just PNG's filtering. PNG filters attempt to tune
   the data to the compressor by finding similarities in the data (like a row
   of pixels being similar enough to the previous row). OpenZL seems to try to
   find similarities and rearrange the data. I'll investigate if it can
   understand things like pixel rows as far as similarities. That seems to be
   a bit of a higher level concept.


On Tue, Oct 7, 2025 at 10:23 AM Chris Blume (ProgramMax) <
programmax@gmail.com> wrote:

> As part of our 5th Edition work, we're investigation options for better
> compression.
>
> By pure chance, Meta just released OpenZL, a long with a blog post and
> whitepaper yesterday:
>
>
> https://engineering.fb.com/2025/10/06/developer-tools/openzl-open-source-format-aware-compression-framework/
>
> I haven't read it yet. I'm sharing with others early.
>