Re: CONTINUATION proposal w/ minimum change

On 1 July 2014 14:14, Jason Greene <jason.greene@redhat.com> wrote:

>
> On Jun 30, 2014, at 6:47 PM, Mike Bishop <Michael.Bishop@microsoft.com>
> wrote:
>
> > You're currently putting the language into the HTTP/2 spec, which
> otherwise knows nothing about the operation of HPACK.  HPACK would need to
> have a concept of a non-state-modifying frame, plus the concept of emitting
> the reference set at the end of the HEADERS/PUSH_PROMISE frame rather than
> at the end of the entire headers block.  But I'm more thinking about
> changes in HPACK *implementations* which are currently libraries that
> operate completely independently of the framing.  This would require what
> is effectively framing information to be passed in/out of those libraries.
>
> Sure there is definitely non-trivial implementation changes required.
>
> >
> > Also, the fact that CONTINUATION is flow-controlled has ramifications
> for both sides.  For the encode, your text says that CONTINUATION can't
> *change* state, but if there are multiple CONTINUATIONs, they can't *use*
> state either.  If I have a huge header to send, and the window doesn't
> permit me to send the whole thing, then I can't reference the static table
> at all without blocking all header compression on the flow control window
> for that stream.  That's not too bad, since saving the size of the header
> name on a >16KB header is relatively small overhead.
>
> Keith’s proposal limits everything to literals. I’m not sure if he meant
> to exclude static table entries with a size 0 table or not (I added a
> comment to this effect). In both cases, the header table is always empty.
> In the latter you can still reference static entries but you get no
> differential encoding. So there is nothing to block on. I do think it was
> unintentional that the wording in Keith’s proposal allows muxing on the
> same stream. I don’t think that makes a lot of sense to do.
>
> Mathew had mention an option in a past email that involved a stream local
> hpack state that would create something to potentially block on, but if you
> establish that CONTINUATIONS don’t mux on the same stream, then even that
> option wouldn’t cause a problem.
>
>
I think I said "state", not "hpack state."​ I meant buffering any partial
headers that carry over from CONTINUATION to CONTINUATION. Sorry if that
wasn't clear. (The end result is similar, though.)

​While this proposal aims to be uninvasive, it's effectively introducing
the concept of two HPACK profiles, which may or may not be a big​ deal
code-wise, but is a big conceptual change.

I suspect it would be less disruptive to completely replace CONTINUATIONS
with a whole new ​frame with its own stand-alone internal encoding. Hell,
if the concatenated payloads of the CONTINUATION frames was just a
traditional "h1: v1\r\nh2: v2\r\n\r\n" string, that'd get the data out
there at least. It does mean either sitting on the header block from HPACK
and then adding the continuated headers, or forcing the application to
accept partial/streamed headers.

Mike:

> >
> > But on the receiver side, the implication is worse.  DATA frames have
> the nice property of being divisible -- if you don't have enough window to
> send the whole thing, you can segment the DATA frame and send as much as
> your window allows.  On the remote peer, that gets passed to the app, and
> as the app reads and frees up memory, WINDOW_UPDATES can be sent to receive
> the rest.  CONTINUATION frames have the same property of being divisible,
> but not the same property of being streamable without major API changes.
>

That's true if you're communicating with someone that doesn't support
header streaming (i.e. anything designed to work with HTTP/1.1, be it a
network component or an application or whatever); but if you're talking
HTTP/2 out both ends (e.g. as a proxy, or part of a new from-the-ground-up
application stack) there's no such restriction.

Theoretically, if the data window and the headers window were controlled
separately (and you could set the headers window size to "infinite") would
that make the idea of flow-controlled headers more acceptable? I say
theoretically because I realise that's even *more* change.

​For the record, I don't necessarily care about header flow control, but I
like being able to progress other information on a connection even while
large/slow headers are oozing their way down one stream's tubes.



> >
> > We're trying not to make invasive changes at this point, and this feels
> a bit large.  I'm much more sympathetic to Willy's proposal,
>
> Sure it’s unfortunate that this is late, but cementing the existing
> CONTINUATION design that “nobody likes”, and will even go unimplemented by
> several products, into the most widely used internet specification seems
> far, far worse.
>
> Is anyone even remotely confident that CONTINUATION, as it is currently
> described,  will survive h3, let alone wide deployment in h2?
>
> >  which doesn't change the wire format but helps discourage CONTINUATION
> abuse by only allowing them when the preceding frame was the max-size.
>  That permits them in the real use-case, but lets someone identify a peer
> that's being overly silly.
>
> I agree its certainly better.
>
>
Assuming you're continuating because the metadata is big, that's fair
enough and I agree; but if there's a header that takes a long time to
generate (say you have a religious preference for content-length over
T-E:chunked, or something), it might be nice to send what headers you
already have, then sleep the current stream while the rest of the headers
are generated. The other way to do that is to force the sender to buffer
the outgoing headers, which is no better than forcing the receiver to do
the buffering. Just a theoretical there, BTW; I got over chunking years ago.


-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/