RE: CONTINUATION proposal w/ minimum change

Responding to everybody at once...



> Martin said:

> I think that there needs to be some text regarding the crossover between

> HEADERS and CONTINUATION. Particularly the case where you have an opcode

> that is followed by a literal that crosses into a CONTINUATION. This needs

> to be clarified. That is, the opcode and all of the literal values

> associated with the opcode MUST fit within the initial frame.



I think this is the easiest solution (i.e. not allowing splits across HEADERS to CONTINUATION boundary). But as Michael points out, you have to at least allow splits across the CONTINUATION to CONTINUATION boundaries in case of a single header >64KB.



What is preferred?

1. No splits across the first boundary; splits allowed on CONTINUATION to CONTINUATION boundaries.

2. Splits allowed on any boundary, but splits across the first boundary must be a non-state-changing opcode.





> Martin said:

> ... the reference set is emitted at the end of the HEADERS/PUSH_PROMISE

> frame, even if there is a partially completed opcode in flight.



Yes, that was a mistake. See above regarding “opcode in flight”.





> Mike & Jason & Herve said:

> Did you intend to allow static table references? Those do not affect

> compression state when the effective table size is 0...



Yes, I completely forgot about static references.  I will definitely add that.





> Herve said:

> ...CONTINUATION frames do not affect compression state, but may use it:

> in HPACK a literal header representation can either use a literal or a

> reference for the header name.



But only static references right? If you reference something that's not static it could be changed by a HEADERS frame on another stream or am I missing something?





> Michael said:

> It feels very awkward to make the compressor aware of framing.  Previously,

> we had said that you take your header set, run it through HPACK, then frame

> up and send the output using HEADERS/PUSH_PROMISE and CONTINUATION.  Now,

> the HPACK compressor has to know where the boundary is and change its

> behavior when it passes that boundary.  I assume that means on the

> decompression side that the reference set is emitted when the frame boundary

> is hit, which means the decompressor also needs to be aware of framing.



These are really good points and trade-offs of this proposal. Jason had reasonable ideas for dealing with them, but in general my preference is to also have loose coupling between layers.



I will point out though that having CONTINUATION frames at all is essentially a coupling between the layers.  If it weren't for the shared hpack state, we would just have HEADERS and DATA frames.





> Michael said:

> With this proposal, I can use flow control to control how much header data

> I receive at once, but I can't drain that filled space by sending it to the

> app, because I'm still waiting for the rest of the header.  That means a

> server can't use "application read" as the trigger to send a WINDOW_UPDATE,

> because it's not passing anything to the app, just realizing that there's

> more to read in before it can, and RTTs are burning while that happens.



Also a really good point.  However, I'll point out again that CONTINUATIONS are supposed to be the exception rather than the rule. According to Richard's data, only 0.02% of requests and 0.006% of requests have headers >16KB. If the performance is worse for large header sets, that might be a good thing - it acts as a disincentive to use large headers.





> Jason said:

> I do think it was unintentional that the wording in Keith's proposal allows

> muxing on the same stream. I don't think that makes a lot of sense to do.

> & Martin said:

> I think that you still need text here to point out that the current stream

> cannot be continued until the header block is complete. That is, you can't

> send PUSH_PROMISE [-END_HEADERS] then send

> DATA/HEADERS/PUSH_PROMISE, you have to send

> PUSH_PROMISE [-END_HEADERS], CONTINUATION...[+END_HEADERS]

> before sending DATA.



Yes, that was a mistake.  On that topic though, I'll point out that one of the more confusing parts of the h2 draft for me is the sequence of frames.

For HTTP it only makes sense to have:

HEADERS [CONTINUATION] DATA [HEADERS [CONTINUATION]]

...but it seems unclear to me if HEADERS and CONTINUATION are allowed to be randomly interspersed with the DATA frames. If so, for what? Metadata? Must the metadata be formatted in name-value pairs and encoded with hpack?



(Also a question regarding what you said Martin. Isn’t there another HEADERS [CONTINUATION] sequence for the response headers after PUSH_PROMISE?)





> Matthew said:

> I suspect it would be less disruptive to completely replace CONTINUATIONS

> with a whole new frame with its own stand-alone internal encoding. Hell,

> if the concatenated payloads of the CONTINUATION frames was just a

> traditional "h1: v1\r\nh2: v2\r\n\r\n" string, that'd get the data out



I really like the length-prefix of hpack.  I'd rather not go back to this.





**Is it worth continuing with this proposal?**



I think Michael brought up some really valid points.



Assuming the following changes:

+ opcodes and their associated literal values MUST fit within the initial HEADERS frame

+ opcodes and their associated literal values MAY span CONTINUATION frames

+ static table references are OK in CONTINUATION

+ same-stream muxing between HEADERS and CONTINUATION is disallowed

+ reference set emitted at the end of HEADERS/PUSH_PROMISE



Does anyone thing this proposal is still worth pursuing?



This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.