Re: H2 vs HPACK header table size... again

Hi Willy,

My interpretation has always been that if the value has *changed* the encoder sends a size update.  The value changes at an encoder when that endpoint sends a SETTINGS ACK for a SETTINGS that had a different value to the existing value.  The value starts at the initial 4k value.

Obviously, this is a bit silly in a few cases, but the cost is marginal.

* If the decoder opens with a >4k value in its preface and the encoder wants to keep the 4k initial value, then the encoder has to send a size change instruction.
* A server rarely has a situation where the initial value is needed as it always has the value from the client preface, but it has to send the SETTINGS ACK and size change instruction anyway, unless the client retains the 4k initial value.

Tatsuhiro's interpretation here matches my own, I think: https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0499.html

On Thu, Dec 23, 2021, at 02:11, Willy Tarreau wrote:
> Hi all!
>
> It was already mentioned long ago but never really settled, there is an
> ambiguity in the HPACK and H2 specs regarding the header table size a
> decoder assumes the encoder will use.
>
> Today I got an haproxy user report an issue with the nghttp client when
> forcing the header table to zero, because nghttp's decoder expects a table
> size update from the encoder while the encoder considers it only conforms
> to what was advertised and never changes it, thus never updates it.
>
> For those who want a bit of background, the report was made here:
>    https://github.com/haproxy/haproxy/issues/1498
>    https://github.com/nghttp2/nghttp2/issues/1660
>
> I discussed a bit with Tatsuhiro about this, who agreed that it was best
> to bring the issue here. We're not trying to decide who's right or wrong,
> and we're both fine with updating our implementations to improve
> interoperability but it's annoying to bindly have to do something we
> don't seem to be reading in the spec.
>
> The H2 spec says about the SETTINGS frame:
>
>    SETTINGS_HEADER_TABLE_SIZE (0x1):  Allows the sender to inform the
>       remote endpoint of the maximum size of the header compression
>       table used to decode header blocks, in octets.  The encoder can
>       select any size equal to or less than this value by using
>       signaling specific to the header compression format inside a
>       header block (see [COMPRESSION]).  The initial value is 4,096
>       octets.
>
> Note the word "initial" here which may or may not participate to the
> confusion. My reading is that until SETTINGS_HEADER_TABLE_SIZE the
> value is 4096, and that SETTINGS_HEADER_TABLE_SIZE changes that value
> to the newly advertised one.
>
> The HPACK spec says in RFC7541#4.2:
>
>    Protocols that use HPACK determine the maximum size that the encoder
>    is permitted to use for the dynamic table.  In HTTP/2, this value is
>    determined by the SETTINGS_HEADER_TABLE_SIZE setting (see
>    Section 6.5.2 of [HTTP2]).
>
> Thus for me it means that the value learned from the H2 layer has to
> be used at the HPACK layer.
>
> Then it continues:
>    An encoder can choose to use less capacity than this maximum size
>    (see Section 6.3), but the chosen size MUST stay lower than or equal
>    to the maximum set by the protocol.
>
>    A change in the maximum size of the dynamic table is signaled via a
>    dynamic table size update (see Section 6.3).  This dynamic table size
>    update MUST occur at the beginning of the first header block
>    following the change to the dynamic table size.  In HTTP/2, this
>    follows a settings acknowledgment (see Section 6.5.3 of [HTTP2]).
>
> Thus my understanding is that an encoder that chooses to use a different
> value than the one learned from the H2 layer has to notify it via an
> update.
>
> And I think this is where there are two different readings which possibly
> depend on how the implementation is built:
>
>   - if you write some client code, it seems natural that HPACK is
>     instantiated before the H2 layer, and in this case I wouldn't be
>     surprised that the "initial" term used above is interpreted as
>     "first set your HPACK default value to 4096 and announce it if
>     you learn otherwise, including from the H2 SETTINGS frame",
>     which seems to be how Tatsuhiro and Pat have interpreted it in
>     the thread below (do not hesitate to correct me, I'm not trying
>     to put words in your mouth at all):
>
>       https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0586.html
>
>   - if you're writing some server code, it seems natural that H2 is
>     instanciated before HPACK and that HPACK comes with the negotiated
>     settings. After all we could even imagine a SETTINGS parameter to
>     switch to a totally different encoder. In this case HPACK starts
>     with an initial value that matches the one advertised by the client,
>     which seems to be what Cory, Hervé and I have interpreted in the
>     thread below:
>
>       https://lists.w3.org/Archives/Public/ietf-http-wg/2015OctDec/0107.html
>
> I do not care about systematically emitting one extra byte to update
> a table size that we already know if that improves interoperability,
> but I find it pretty annoying that 6 years later we continue to face
> interoperability issues because there seem to be two incompatible
> readings of this corner case of the spec. And I suspect that the issue
> also exists for situations where a large SETTINGS_HEADER_TABLE_SIZE
> value is emitted by the decoder, the encoder is not willing to change
> from 4kB and stays at 4kB, but the decoder assumes that the encoder
> agreed with the initially advertised one. This can be worse because
> it could work without triggering compression errors but possibly
> result in damaged messages.
>
> As such I'd like that we could collectively decide what to do for
> this (even if that means relaxing both clients and servers to
> accommodate for the other interpretations that are already deployed).
>
> And maybe once we're fixed with whatever possible interpretation, we
> should possibly file an errata to amend this part of the HPACK spec
> in a way that doesn't leave any more room for two interpretations.
>
> What do implementers (or others) think ?
>
> Thanks,
> Willy

Received on Wednesday, 22 December 2021 22:24:53 UTC