Re: Broader discussion - limit dictionary encoding to one compression algorithm?

On Tue, May 21, 2024 at 01:02:30PM -0400, Patrick Meenan wrote:
> On Tue, May 21, 2024 at 12:41 PM Poul-Henning Kamp <phk@phk.freebsd.dk>
> wrote:
> 
> > Patrick Meenan writes:
> >
> > > ** The case for a single content-encoding:
> > > […]
> > > ** The case for both Brotli and Zstandard:
> >
> > First, those are not really the two choices before us.
> >
> > Option one is:  Pick one single algorithm
> >
> > Option two is:  Add a negotiation mechanism and seed a new IANA registry
> > with those two algorithms
> >
> > As far as I can tell, there are no credible data which shows any
> > performance difference between the two, and no of reason to think that any
> > future compression algorithm will do significantly better.
> >
> 
> We already have a negotiation mechanism.  It uses "Accept-Encoding" and
> "Content-Encoding" and the existing registry. Nothing about the negotiation
> changes if we use one, two or more. The question is if we specify and
> register the "dcb" content-encoding as well as the "dcz" content encoding
> as part of this draft or if we only register one (or if we also add a
> restriction that no other content encodings can use the dictionary
> negotiation).
> 
> As far as future encodings, we don't know if any algorithms will do better
> but there is the potential for content-aware delta encodings to do better
> (with things like reallocated addresses in WASM, etc). More likely, there
> will probably come a time where someone wants to delta-encode
> multi-gigabyte resources where the 50/128MB limitations laid out for "dcb"
> and "dcz" won't work and a "large window" variant may need to be specified
> (as a new content encoding).

A practical approach is to allow for future unknowns as you describe
above, and to pick one of (brotli, zstd) to be required to be
implemented in this version of the standard, with the other optional.
Future versions might have a different required algorithm, and include
the intention that an algorithm required in a prior version will remain
an option in future versions.

If a content-provider spends the time to build procedures and
infrastructure to deploy compression dictionaries, they will probably
experiment with their content and their CPU, memory, and other resource
limitations; and along with client capabilities, balance their choices
based on all of those inputs and more.

Proxies and CDNs may make different choices on what they support
receiving from origin servers, how they store it, and what they support
sending to clients.  Intermediate security scanners and possibly
alternate corporate malware will add other limitations.

Compression dictionaries are an optional feature and one that an origin
server might choose not to implement (or a server deployed before and
older than the specification).

Cheers, Glenn

Received on Tuesday, 21 May 2024 18:01:58 UTC