Digest Headers and Structured Fields

Hi folks,

You might recall that the Digest Headers draft WGLC concluded in December.
We had some feed that we’ve been digesting (pun intended) and wanted to
bring some of our ideas on next steps back to the WG for input. There’s a
few options here and we’d appreciate feedback so that we can whittle them
down. Apologies for the wall of text, it's intended to bootstrap discussion

The crux of the issue is that this draft relates to updating the Digest and
Want-Digest HTTP fields. These use a “custom” list format based on the
#rule ABNF extension [1]. This is more-or-less unchanged in the draft. List
items are of the form “key=value”. This format is very similar to
Structured Fields (SF) Dictionary but not exactly the same. It supports a
set of characters that do not match SF’s set , and it allows for duplicate
entries in the list (handling of duplicate entries is a bit under-defined,
we’ll come on to that). In 2021 we added the Content-Digest and
Want-Content-Digest headers. These use the same format as Digest and
Want-Digest for consistency.

A few people have expressed that they’d really like to find a way for
Digest headers to work with SF. That seems to be the direction of travel
for new HTTP fields. Digest synergizes with another WG item - Signature,
and it’s unfortunate that there’s a bit of friction between the two due to
formats. But we’re also updating an old draft and need to be mindful of
existing deployments that use Digest. The catalyst for starting the Digest
work was to replace “Instance” with “Representation” and other modern HTTP
semantic terms so that implementers had a clearer understanding how to use
Digest as intended.

We’ve been noodling on things in the background and think we have 3
candidate options for consideration.

* Option 1: Status Quo. The I-D continues to update RFC 3230 terms, and has
a light touch on Digest. New Content-Digest uses legacy list format. This
addresses the initial goal and accommodates the simpler content use case
that came up during standardization.

* Option 2: “Three headers”. Update RFC 3230 terms and light touch on
Digest. New Representation-Digest and Content-Digest headers use an SF
Dictionary format. Digest and Representation-Digest act similarly but use
different wire formats and have different parsing rules around duplicates;
new headers are RECOMMENDED over old. See the proposal PR at
https://github.com/httpwg/http-extensions/pull/1893; diff at
https://www.ietf.org/rfcdiff?url1=https://httpwg.github.io/http-extensions/draft-ietf-httpbis-digest-headers.txt&url2=https://httpwg.github.io/http-extensions/structured-digest2-and-content-digest/draft-ietf-httpbis-digest-headers.txt

* Option 3: “Two headers”. Leave RFC 3230 alone. New Representation-Digest
and Content-Digest headers use an SF Dictionary format. Digest and
Representation-Digest act similarly but use different wire formats and have
different parsing rules around duplicates; new headers are RECOMMENDED over
old. See the proposal PR at
https://github.com/httpwg/http-extensions/pull/1894; diff at
https://www.ietf.org/rfcdiff?url1=https://httpwg.github.io/http-extensions/draft-ietf-httpbis-digest-headers.txt&url2=https://httpwg.github.io/http-extensions/structured-digest2-and-content-digest-no-rfc3230/draft-ietf-httpbis-digest-headers.txt

Note that we have omitted Want-Digest from this email for brevity. A
complete comparative table of the proposals is available on a Google doc[2].

Note that the proposal PRs are a bit rough. They are more there to give a
shape of what the differences would look like. We can probably wordsmith
them, and bikeshed on whether to use sf-dictionary or sf-list and so on.
But the important thing is to agree on what to do first. So feedback is
appreciated.



Cheers
Lucas & Roberto
Digest editors


[1]
https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-semantics-19#section-5.6.1
[2]
https://docs.google.com/document/d/1wC2E42dCSrmh3nDnUL-rItzvr09kFcltBs_jWvJPfjI