Re: Fragmentation for headers: why jumbo != continuation.

Unfortunately not true-- the backend can't have the same compressor state
as the proxy unless the proxy is doing a 1:1 connection mapping (which only
happens sometimes).
If the proxy is handling 2 FE connections and 1 BE connection, or 1 FE
connection and 2 BE connections, there can be no safe delegation of
compression to the backend, since it cannot know the state of the
compression context at the proxy.

If you need to know the size of the headers, your function will look
something like:


# Non-streaming forwarding
  header_frame = read_header_frame()
  uncompressed_headers = decompress(compressed_headers)
  outgoing_compressed_headers = compress(uncompressed_headers)
  send_header_frame(outgoing_compressed_headers)

If you don't need to know the size of the headers, you can instead do:

# Streaming forwarding. ~ 6 LoC more.
header_payload = []
while True:
  header_frame = read_header_frame()
  header_payload.append(header_frame.payload())
  partial_uncompressed_output, amount_consumed = decompress(header_payload)
  header_payload = header_payload[amount_consumed:]  # state is freed up
here while sending
  compressed_outgoing = compress(partial_uncompressed_output)
  send_header_frame(compressed_outgoing)
  if header_frame.has_end_headers():
    break

Note that in the case above, neither the full set of incoming headers, nor
the full set of outgoing headers needs to be resident in memory before
forwarding.
A decision to forward like above does involve a tradeoff between lessening
memory commitment (and latency) and the possibility of HOL blocking. This
should be a choice that an implementation makes. Some implementations will
know that the connection between the backend and a proxy is always faster
than the connection from the user to the proxy. In such cases, HOL blocking
will almost never occur, and streaming the headers to the client wins both
latency and memory reductions at almost no cost.

-=R


On Sun, Jul 13, 2014 at 11:14 AM, Poul-Henning Kamp <phk@phk.freebsd.dk>
wrote:

> In message <CAEn92TrULC1rniqaG-vFYw9K77d2g97=ANywW3x5UyPfPSR=
> jA@mail.gmail.com>, Johnny Graetting
> er writes:
> >--20cf3079bc781d287e04fe16336c
> >Content-Type: text/plain; charset=UTF-8
> >
> >On Fri, Jul 11, 2014 at 7:53 AM, Amos Jeffries <squid3@treenet.co.nz>
> wrote:
> >
> >> Given the size of a large frame up-front a proxy can actually stream the
> >> start of that frame before the end has arrived if you trust the sender
> >> will complete it cleanly.
> >>
> >
> >The proxy cannot know the encoded size of the block until it's run that
> >complete block through it's send-side HPACK context. That context is
> >different than the recv-side context, and it will almost certainly result
> >in a different block length.
>
> actually, if there exists such a intimate relationship between the
> backend and the proxy, the backend can compress the headers to make
> that possibl.
>
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk@FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
>