Re: CONTINUATION proposal w/ minimum change

On Jun 30, 2014, at 4:49 PM, Mike Bishop <Michael.Bishop@microsoft.com> wrote:

> Op-code splits are required -- if a single header can be up to 64KB, and the limit of a single frame is 16KB, then just to encode that header will cross several frames.  

Right, large headers would get kicked into another frame. Ideally you would do that before even encoding. As a mental model picture having two header sets. A smaller efficiently compressed set that fits in a frame and represents 99.9% of the internet, and the overflow set which contains the exceptional and legacy content which is split over N CONTINUATION frames.

> If you're simply saying that op-codes can't cross the *first* boundary, that may be possible,

Yes that’s exactly what I meant.

> though it means that a very small HEADERS frame could still have a CONTINUATION.


Sure, if you had say 4-5 common routing fields, and then a large kerberos ticket, that would be two frames with a small header. This would basically be the reverse of the current draft which would have a max HEADERS frame and a small CONTINUATIONS frame. IMO, this is actually beneficial because the state affecting frame will be faster to process than the non-state-affecting.

> 
> At any rate, my biggest piece of feedback remains -- this isn't a minimal change.  It's going to require pretty substantial changes in HPACK and how it interfaces with the remainder of the protocol.

I don’t see how the HPACK spec needs to be touched, but maybe I am missing something?

> 
> -----Original Message-----
> From: Jason Greene [mailto:jason.greene@redhat.com] 
> Sent: Monday, June 30, 2014 12:45 PM
> To: Mike Bishop
> Cc: K.Morgan@iaea.org; mnot@mnot.net; phk@phk.freebsd.dk; ietf-http-wg@w3.org
> Subject: Re: CONTINUATION proposal w/ minimum change
> 
> 
> On Jun 30, 2014, at 1:53 PM, Mike Bishop <Michael.Bishop@microsoft.com> wrote:
> 
>> It feels very awkward to make the compressor aware of framing.  Previously, we had said that you take your header set, run it through HPACK, then frame up and send the output using HEADERS/PUSH_PROMISE and CONTINUATION.  Now, the HPACK compressor has to know where the boundary is and change its behavior when it passes that boundary.  I assume that means on the decompression side that the reference set is emitted when the frame boundary is hit, which means the decompressor also needs to be aware of framing.
>> 
>> It gets worse, because a given instruction could cross the boundary.  Whether a header can be indexed or not would depend on whether the header name is already in the table, because the compressor needs to check whether a given header will cause it to cross the boundary.  There's new HPACK text here that will be required, and not very pretty.
> 
> You could just have the hpack routines be reusable, and have the compressor accept an output length limit. Once the limit is hit, the compressor backs up and informs the caller where it left off. When an overflow occurs the caller then invokes the generic hpack routine again at a resume point using zero-compression settings (literals, restricted table size etc). For simplicity, I think its important to ensure that opcode splits are not allowed.
> 
> This of course could be much more intelligent. If we look at the primary use-case of a kerberos ticket, the ideal implementation would sort the gigantic headers into the second uncompressed call, likely resulting in no actual overflow on the first call.
> 
> You can look at decompression the same way. The generic hpack routine gets a HEADER frame, it completely decodes it, and emits a reference set because for all it knows, it's done. Subsequent hpack decodes do not use compression state, so the reference set need not be touched. 
> 
> Alternatively you could opt for a less protocol efficient input limit check, and simply split when your input is > 16K, leaving your hpack routine largely unaltered.
> 
>> 
>> By the way, I thought the point of making CONTINUATION only carry literals was to break the requirement that a CONTINUATION MUST immediately follow the incomplete HEADERS/PUSH_PROMISE frame - but I don't see that your pull request does that.  You've brought in new ugliness without getting the return you're looking for.
>> 
>> (Having now read Martin's comments, there's a lot of overlap here.)
>> 
>> From: K.Morgan@iaea.org [mailto:K.Morgan@iaea.org]
>> Sent: Monday, June 30, 2014 11:00 AM
>> To: mnot@mnot.net; phk@phk.freebsd.dk
>> Cc: ietf-http-wg@w3.org
>> Subject: CONTINUATION proposal w/ minimum change
>> 
>> On Friday,27 June 2014 09:36, mnot@mnot.net wrote:
>>> Can someone please provide a proposal?
>> 
>> I've incorporated many of the ideas mentioned on the list and in the NYC minutes to create a proposal that (I believe) accomplishes the goals with a minimum amount of change.
>> 
>> Changes:
>> + Only the HEADERS and PUSH_PROMISE frames can alter the compression context.
>> + CONTINUATION frames can only use hpack literals (optionally Huffman encoded) that don't alter the table.
>>  - CONTINUATION is flow controlled
>>  - CONTINUATION has an END_STREAM flag
>> 
>> Benefits:
>> + Simplifies CONTINUATION implementation
>>  - i.e. fixes the nonsensical END_STREAM that doesn't actually end 
>> the stream
>> + Flow control of CONTINUATION:
>>  - removes the incentive for implementations to cheat and send large amounts of data in the HEADERS+CONTINUATION jumbo frames
>>  - may help with certain HOL blocking situations (e.g. connection 
>> coalescing)
>> 
>> See pull request here: 
>> https://github.com/shearl/http2-spec/commit/5a548553334e35cbba5946d238
>> e9a14c6ba03f7e
>> 
>> 
>> This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.
>> 
> 
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect JBoss, a division of Red Hat
> 
> 

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat

Received on Monday, 30 June 2014 22:16:05 UTC