Re: Compression dictionary draft ID - draft-meenan-httpbis-compression-dictionary-00

That would necessarily couple the use of a dictionary with the negotiation
of the storage. They way it is currently set up, "use-as-dictionary" is one
way to populate the dictionary but the dictionaries could also be
pre-loaded into a client or otherwise negotiated.

As far as state, if you're passively observing traffic to compress and not
actively managing the dictionaries, doesn't something like this work?

1. See response with "use-as-dictionary" response header
2. Store dictionary in key-value store, using the hash of the payload as
the key
3. See request with "sec-available-dictionary" request header

For supporting static, offline resource compression:
- If a version of the resource is in cache compressed with the requested
dictionary, serve it
- If not, kick off an async task to compress the cached full response with
the dictionary (if one is available in KV)

For dynamic responses:
- If the dictionary is in KV, use it to compress the response

The main "state" that needs to be managed is the storage of the dictionary
keys, indexed by the appropriate hash.

If the CDN is generating the dictionaries (or adding the headers) there's
not much more state than that either.

I'm sure I'm probably hand-waving a lot of things that need to be done at
scale when handling thousands of dictionaries and thousands of simultaneous
requests for the same dictionary but I'd hate to add complexity to the
negotiation itself if it isn't absolutely necessary.

On Tue, Aug 8, 2023 at 9:49 AM Vlad Krasnov <vlad@cloudflare.com> wrote:

> I think it would make the proposal easier to support from a CDN POV if the
> Sec-Available-Dictionary included the dictionary URL and Etag, otherwise it
> requires too much state keeping.
>
> Given a URL and an Etag it is pretty easy to have a best effort dictionary
> compression.
>
> Best,
> Vlad
>