Re: clarify some MUST requirements in HTTPbis part 1 section 3.3

Hi Alex,

On Fri, Dec 02, 2011 at 04:12:44PM -0700, Alex Rousskov wrote:
> I divide headers into "violating HTTP specs" and "not violating HTTP specs".

The issue I have with this is that for me, violating the spec simply implies
not doing a MUST or doing a MUST NOT. There are a huge number of such rules
in the spec, many of them irrelevant to most proxies. And by ignoring these
rules, the proxies will violate the spec by forwarding wrong contents. Your
example of the Date header is perfect. It's a general header with a MUST for
the format, still a number of proxies don't care about it and will not check
it. By forwarding a wrong one, they will violate the spec. The issue I'm
seeing is that by making it "normal" to violate the spec for a number of uses, 
critical rules are not necessarily spotted. That's why I like your proposal
which specifically focuses on the Content-Length header.

> >> If some specification defines an extension HTTP header that requires
> >> special forwarding beyond RFC 2616 rules, that specification itself is
> >> broken.
> > 
> > It's not that simple. There are issues which only happen when forwarding.
> > For instance, I once used to add "Connection: x-forwarded-for" to my
> > requests to some reverse-proxies to show that my IP address was not
> > passed to the origin server. This is a typical example of something
> > which only affects forwarding. X-Forwarded-For is not part of a spec
> > but you can do that with other headers and observe surprizing results.
> 
> Sorry, I do not see how your example contradicts what I said. Correct
> handling of the Connection header is specified in RFC 2616. Connection
> header value may include extension headers so adding X-Forwarded-For to
> Connection is fine as far as RFC 2616 is concerned. A compliant proxy
> receiving such a Connection header would have to remove the
> X-Forwarded-For header from the received request, even if it knows
> nothing about that particular extension.

The problem is the order in which it is done. It is said "before a message
is forwarded". Some proxies remove the headers they have added themselves.
So you have a first layer of reverse proxy in front of your servers, which
adds the client's IP address in X-Forwarded-For for logging purposes, and
which immediately removes it due to the Connection rule. Connection applies
to the headers received by the proxy, not to headers sent. But as the text
says "forwarded", this is ambiguous. The example with Transfer-Encoding is
similar and nastier since it can lead to content smuggling, though I'm not
aware of a proxy which applies Connection without fixing Content-Length
and/or Transfer-encoding.

Regards,
Willy

Received on Saturday, 3 December 2011 06:53:56 UTC