Re: Pipelining in HTTP 1.1

Steingruebl, Andy wrote:
> Jamie Lokier wrote:
> > Roy T. Fielding wrote:
> > > or even extend HTTP
> > > such that the request/response contains some form of matching ID.
> > 
> > I did this in an application-specific extension of HTTP, and learned
> > something not obvious to me.
> > 
> > Beware if you allow reordering and really use it, you can get into
> > deadlock or unbounded-buffer-needed scenarios, unless you also
> > implement some kind of per-ID flow control.
> > 
> > I found interleaving HTTP-style chunks, labelled with a
> > request-response ID, and per ID receive windows, works well.  Many of
> > the annoying problems of HTTP pipelining go away.
>  
> Out of curiosity did you just use sequential ID's to track this, or
> something more like a GUID?

I use client-selected IDs per request, and I let them be alphanumeric
(token syntax).  So you could use GUIDs if you wanted, but my
implemention just uses numbers :-)

Sequential numeric IDs are simpler, but I didn't want to imply an
arbitrary limit where they wrapped around or something.

The client is free to reuse IDs which are no longer in flight.

In fact the client can reuse IDs which are already in flight, and this
has the effect of forcing the response ordering to be retained, the
same as HTTP pipelining (just for that ID).

Absence of an ID is treated as just another ID value, so the ordering
rule means it degrades gracefully to HTTP pipelining when you don't
send the ID header.

> Several folks have been toying around with
> protocol-level changes to mitigate/prevent Response-Splitting attacks
> and being able to tie request+response is critical.

That's a different requirement, but you can obviously use the same
mechanism.

I don't have response-splitting attacks in my application which uses
this ID extension because (a) it's all private stuff at the moment,
and (b) every message has a clearly defined length, set in a header or
a chunk header.

I suspect avoiding Response-Splitting cannot be solved by tying
requests to responses with a simple ID.  If you can fake protocol in
the body, you can fake IDs too.  Randomised IDs (the reason you might
be using GUIDs, but the important part is unpredictability isn't it?).

But even randomised IDs won't prevent attacks which trick a receiver
into receiving some chunks of a different response as content, parsing
it on the receiver (e.g. in Javascript in a browser) to get the other
response's ID, sending that ID somewhere malicious, which then knows
the ID to attach to other chunks for malicious injection later in the
same stream.

If the protocol level is to be modified, then just do it properly: fix
pipelining to do out-of-order chunk multiplexing properly and safely
with clearly defined message boundaries, probably using chunks.  Really.

> Not that webapps should have this vulnerability, but it wouldn't really
> be possible with strict 1:1 correlation between request and response.

Indeed it's not.  But if you're seeing the existing well-defined
message boundary rules subverted, a server or proxy is at fault
somewhere for not enforcing the rules.  If you can't solve that
problem, how will adding ID headers make it work?  At most they'll
improve the reliability of detecting a mistake, but I suspect
malicious attacks will still find ways around it, although they'll be
more difficult.

-- Jamie

Received on Saturday, 4 April 2009 03:16:18 UTC