Re: HEADERS and flow control

​On 26 May 2014 00:46, Yoav Nir <ynir.ietf@gmail.com> wrote:
>
> On 23 May 2014 03:28, Matthew Kerwin <matthew@kerwin.net.au> wrote:
>>
>> On 23 May 2014 04:08, Greg Wilkins <gregw@intalio.com> wrote:
>>>
>>> I think the hpack tail is really wagging this protocol dog!
>>>
>>> Currently hpack is forcing this protocol to have a user accessible
>>> non flow controlled un-segmentable size unlimited meta data streams
>>> that has strict ordering criteria resulting in significant parallel
>>> slow down.
>>>
>>
>> ​It's also the single most complex part of the spec​ to implement. My
>> side-project implementation has hiccupped and stalled every time it's
>> bumped into continuation frames and hpack. I've currently stopped all
>> work on it until I have time to go through and read in much more
>> detail the hpack draft, and some related archived conversations on
>> the list, and Canonical prefix encodings and Huffman coding, etc.
>>
>> If I could negotiate not using HPACK I could probably have the rest
>> of my implementation up to some sort of interop testing by now.
>>
>
> I might be missing something basic here. But if you set the
> SETTINGS_HEADER_TABLE_SIZE to zero, doesn’t this effectively eliminate
> compression (and state!), leaving only encoding? ISTM that it works
> for a minimal client.

I'm not sure, that's one of those things I'd have to infer from a solid
reading of the spec, or find in the mailing list archives. I think
it's correct.

> Am I correct in assuming that compression and state are the painful
> parts?

I don't know that there are any painful parts, per se; more that there's
a whole separate 17 page spec, with 8 pages of appendices, that requires
20 pages of examples, _and_ that the streaming compression adds other
requirements and limitations to HTTP2 itself.

In terms of implementation, so far the header table is pretty easy, and
I think I've started to get a handle on the reference set; but I've
never written a Huffman coder before, nor even heard of canonical prefix
encoding. My networking experience comes from a place where RLE was
enough.

>
> A minimal server could potentially receive a bunch of requests with
> compressed headers before the client receives the SETTINGS frame. But
> I’m sure that could be worked around, for example by closing all
> streams that have compressed headers. Not quite efficient, but if
> efficiency was the primary goal for this hypothetical server, it would
> implement HPACK.
>
> What am I missing?
>

The first hurdle was when I realised that CONTINUATION frames have to be
contiguous, so my low-level architecture went from something resembling
a simple packet handler to a hybrid with the ability to effectively
block on subsequent packets.  That wasn't even an issue in the HPACK
spec, but an implication of using HPACK in HTTP/2.  Fortunately in that
case I was able to fall back on an earlier decision I'd made (out of
laziness), where I could block waiting for an ACK after sending the
initial SETTINGS.

My next issue, assuming HPACK itself is implemented correctly (or
someone gives me an awesome library, or something), is that I'll have to
redesign parts of my low-level machinery because, for the first time in
the protocol, the actual value of some received message isn't contained
inside the frames themselves. So my packet switching architecture
isn't able to discuss headers with higher-level abstraction layers.
I have an idea of how to proceed, by effectively jamming the HEADERS
(or PUSH_PROMISE) and all its associated CONTINUATIONS into one big
superframe, but it's a bit of a work around, and incidentally
complicates other earlier design decisions.

HTTP/1.1 was, generally, easy at this level, because it's a piece of
cake to read up to a ":", and then again to a "\r\n" (usual caveats for
LWS, folding, quoted strings, etc.). Then you emit the pair, and repeat.
HTTP/2 was almost easy, because I was almost able to deserialise frames
one by one and emit them as they arrived, except that I can't do that
for headers (or push_promise).


-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/