Re: clarify some MUST requirements in HTTPbis part 1 section 3.3

On 12/01/2011 03:32 PM, Patrick McManus wrote:
 > proxies are clients, and I'd like to think the spec aspires to requiring
> all clients to send well formed messages. so I would favor mandatory
> cleanup.

On 12/01/2011 06:40 PM, Adrien de Croy wrote:

> I don't see how we can write prose into HTTP that permits any agent to
> send more than one Content-Length header under any circumstances and
> still be compliant with the spec, no matter what you call the software
> that does it or how it operates.


We cannot write such prose, but folks who interpret "send" differently
from "forward" do not see that as an obstacle. We can write prose to
clarify whether "forward" is "send".

For example, when somebody starts complaining that the proxy is not
compliant because it is sending two Content-Length headers, the proxy
developer may claim that the proxy is not sending two Content-Length
headers but just forwarding two end-to-end headers received from the
previous hop. They would say that the proxy has no business fixing
end-to-end headers. I have heard many such arguments, usually backed by
a "do no harm" principle.

If there is consensus that any HTTP proxy is responsible for all
messages it forwards (i.e., send = forward), then HTTPbis should clearly
state that once and for all. For example:

  In the absence of requirements specific to proxies, when forwarding
  an upstream message, an HTTP proxy MUST comply with requirements
  for HTTP clients. Consequently, if the upstream message is
  malformed, a proxy MUST NOT forward it "as is".

If we state that, an HTTP proxy would have to make a choice when dealing
with a "broken" incoming message:

  1) reject the malformed message
  2) forward a "fixed" message
  3) forward raw bytes, becoming a TCP tunnel

Currently, many interpret HTTP specs as if there is another option:

  4) forward the malformed message "as is"
     while interpreting it correctly internally.


Thank you,

Alex.



> On 2/12/2011 12:14 p.m., Willy Tarreau wrote:
>> Hi Patrick,
>>
>> On Thu, Dec 01, 2011 at 05:32:46PM -0500, Patrick McManus wrote:
>>> On Thu, 2011-12-01 at 09:17 -0700, Alex Rousskov wrote:
>>>
>>>> Absolutely, but the question paused by this thread remains: Are proxies
>>>> _required_ to fix messages they forward?
>>> proxies are clients, and I'd like to think the spec aspires to requiring
>>> all clients to send well formed messages. so I would favor mandatory
>>> cleanup.
>> I see it from a different angle. In my opinion, proxies are not client
>> in that they don't author requests but forward what they understood from
>> what they received. It's much easier to be a clean sender when you're a
>> client than when you're a proxy. If we brought the multiple
>> content-length
>> issue here, it's precisely because proxies were facing it. There are many
>> situations where proxies are expected to perform some cleanup based on
>> protocol compliance, and expected to act differently based on client or
>> server bugs that end-users expect them to ignore.
>>
>> I have a recent example. About 2 months ago, I got a report that a
>> haproxy
>> user was sometimes seeing haproxy reject bad requests. Upon
>> investigation,
>> it appeared that a buggy client software was cumulating headers between
>> each request to the point where some headers were sent 37 times before
>> being blocked due to too large a request. Very few users reached this
>> limit and were affected. Fortunately for them, haproxy is not able to
>> fold multiple headers when forwarding, otherwise they would have come to
>> a complete stop because the server-side software's parser was unable to
>> handle a comma in the affected header.
>>
>> I'm not saying that it's better not to fix, and in other circumstances,
>> the opposite could have been true. I'm just saying that in this specific
>> case it was fortunate that the forwarded stream was as little mangled as
>> possible. My principle has always been to strictly control inputs, then
>> don't touch what you don't absolutely need to. From my experience, it
>> has always resulted in the least breakage.
>>
>> However, just like a client, a proxy which is adding or mangling headers
>> has absolutely no excuse for emitting them in a wrong format.
>>
>> Regards,
>> Willy
>>
>>
>

Received on Friday, 2 December 2011 17:24:03 UTC