Re: On the abuse of chunking for interactive usages

On Tue, May 10, 2011 at 2:45 PM, Poul-Henning Kamp <phk@phk.freebsd.dk> wrote:
>
> In message <20110510212615.GA504@1wt.eu>, Willy Tarreau writes:
>
> >My concern comes from this specific use of chunking. I see no reason why
> >this would be valid. I know it will not work at many places. Some proxies
> >(such as nginx IIRC) buffer the complete request before passing it on.
>
> Well, it doesn't say anywhere in RFC2616 that "A server SHALL NOT
> reply until it has received the entire request including the body,
> if any.", so from a standards point of view, it is not "invalid".
>
> But we can readily agree that it will probably come as a major
> surprise to most readers of RFC2616 that anybody would do this.
>
> And most HTTP proxies (as opposed to HTTP load-balancers) will
> not permit it through at all.
>
> You can configure Varnish to let this through:  Varnish has a
> "pipe" mode, where after the first request on a TCP connection has
> been passed to the backend, varnish just moves bytes forth and back.
>
> But that is merely a reflection of my "code, not policy" attitude
> to software tools, I agree with you that this should be discouraged
> as much as possible.

I can imagine variants of Willie's use case that happen even in the
absence of chunking: an origin server receiving a large file upload
might attempt to stream back a response "10% complete ... 20% ..."
incrementally.  Thus I think the general issue is:  Should a
proxy be allowed to (or prevented from, or perhaps even required
to) stream a request though toward the origin server before it
receives the entirety of the request?

I think the "allowed to" option is the most pragmatic approach.
There are proxies that have good reasons for buffering the entire
request body (ease of implementation, scanning of POST payload
for malicous input), and there are proxies that have good reasons
for not buffering the request body (minimization of memory usage,
denial-of-service avoidance).  It seems prudent to let each
implementor decide what tradeoffs are best for their implementation.

-Brian

Received on Tuesday, 10 May 2011 22:13:53 UTC