Re: METADATA frames for HTTP/2 and HTTP/3

Hi Bence,

On Wed, Jul 27, 2022 at 4:30 PM Bence Béky <bnc@google.com> wrote:

> Hi,
>
> Responding to Lucas's concern, it turns out our metadata implementation is
> relatively simple.  Yes, it is a new setting identifier and a new frame
> type, but HPACK/QPACK usage is trivial: a new encoding context is
> instantiated for serialization of each metadata block, and a new decoding
> context for parsing each metadata block (that can be fragement over
> multiple METADATA frames in HTTP/2).  A single encoding/decoding context
> pair could also be used for the entire connection to cut down on object
> instantiation/destruction overhead.  The requirement that the HPACK and
> QPACK encoding is not allowed to use the dynamic table allows for a simpler
> implementation: there is no need to plumb to the shared compression context
> of the connection, or--in the case of QPACK--to the decoder stream.
>
> To respond to Cory's suggestion about making metadata use a separate
> compression context: the reason we don't do that is that for our particular
> use case the keys tend to be short and the values are usually unique, so we
> don't expect to gain a lot from compression.  That being said, there is no
> technical reason not to use a separate compression context and allow the
> use of the dynamic table.  The current draft simply reflects our internal
> use case, and we appreciate any feedback to make this proposal more
> suitable for other people's use cases.
>
> For example, one option could be to introduce something like
> SETTINGS_HEADER_TABLE_SIZE_FOR_METADATA and
> QPACK_MAX_TABLE_CAPACITY_FOR_METADATA instead of the currently proposed
> SETTINGS_ENABLE_METADATA.  Any value would advertise support for METADATA
> frames, the lack would mean no support, and deployments that would not
> benefit significantly from compression could send 0.
>

This is helpful, thank you. It partly addresses my concern - the remainder
is how this capability is communicated and exposed to developers so they
build apps as intended.

The first time I heard about the METADATA frame (IETF 106?), it was in the
same breath as Roy talking about sending HTTP fields during a request and
response; something I'd also heard be called "middlers". The WG didn't
settle on doing explicit work on such things but neither did it seem to
shut the door on them. But what I seem to be hearing in this thread is that
actually the METADATA frame is not intended to relate to HTTP messages, and
instead the use case is quite a narrow, point-to-point communication
between ends that have some arrangement in place. Since it sounds like
METADATA doesn't want HTTP fields to act as fields, the dynamic compression
is prohibited, and the HPACK/QPACK static tables are optimized for actual
HTTP rather than this thing; I'm curious why jump through the hoops of
converting your raw data into HTTP field sections, then encoding them with
HPACK/QPACK. If the key names are short and the values unique - wouldn't
some other binary encoding be simpler? I'm thinking along the lines of
binary HTTP [1], binary structured fields [2], CBOR, or something like that.

I get that the work is a neat reuse of some existing things but I don't
think many of the interop challenges come from the wire format of these
frame content per se.

Cheers
Lucas

[1] -
https://httpwg.org/http-extensions/draft-ietf-httpbis-binary-message.html#section-3.6
[2] -
https://datatracker.ietf.org/doc/html/draft-nottingham-binary-structured-headers-02



>