Re: clarify some MUST requirements in HTTPbis part 1 section 3.3

On Fri, Dec 02, 2011 at 02:12:14PM -0700, Alex Rousskov wrote:
> > Actually, it's even more common to see this :
> > 
> >     5) ignore 90% of headers you don't care about and for which you
> >        don't know how to determine the validity, and apply 4) when
> >        you care about the header.
> 
> Actually, RFC 2616 does define what a valid header is. Any header not
> explicitly defined in that RFC still has to obey the rules for extension
> headers. The proxy does not have to follow any other rules to stay
> compliant.

Some trickier situations have become apparent since 2616. One of them
being the multiple Content-Length. Each of them taken individually is
OK. Having them together is wrong, still it happens and causes folding,
resulting in "100, 100" appearing in the value. How to handle multiple
or empty Host headers is another issue. Another example, if you receive
"Connection: Transfer-Encoding" with both a Transfer-Encoding and a
Content-Length headers. You're supposed to use Transfer-Encoding to
determine the transfer length and ignore the Content-Length, to
remove the Transfer-Encoding header and to forward the message. The
next hop will only use the Content-Length that you ignored. And I
think that since you're a proxy implementer, you're well used to
facing such issues and to have to take decisions at one point.

> > I don't see how we can impose 2) for the 90% headers that a proxy
> > does not understand and just ignores.
> 
> Per my proposed wording, those headers do not need fixing: Any extension
> header can be forwarded "as is" while staying compliant. There are some
> nuances related to handling of multiple extension headers with same
> names, but the same "as is" rule works for those as well.

It's not that black and white. For instance in haproxy I don't use the
Host header, I leave it unaltered so that the next hop gets the real
value. So if there are crappy values there, they will pass through.
It is possible that in your proxy you heavily rely on Host and that
you find it obvious to fix it.

My current practice is to forward everything "as-is" but to be strict on
checks I need to get my work done : if I can correctly parse the header
when applying the rules, then the next hop will be able to do so and that's
fine. Otherwise I reject the message. It still straightens my hair to let
a Content-Length header leave with multiple values but it's particularly
expensive to fix every corner case. Still, I'm ready to fix a few really
dangerous ones, especially if I need to handle them in a specific way.
But I will not do a massive cleanup.

> If some specification defines an extension HTTP header that requires
> special forwarding beyond RFC 2616 rules, that specification itself is
> broken.

It's not that simple. There are issues which only happen when forwarding.
For instance, I once used to add "Connection: x-forwarded-for" to my
requests to some reverse-proxies to show that my IP address was not
passed to the origin server. This is a typical example of something
which only affects forwarding. X-Forwarded-For is not part of a spec
but you can do that with other headers and observe surprizing results.

It's not just a matter of broken specs, just that HTTP has evolved much
farther than initially designed for, inevitably resulting in nasty corner
cases. That's what http-bis is trying to address after all, so I think
that your request for clarifying "Send" vs "Forward" is welcome !

Regards,
Willy

Received on Friday, 2 December 2011 22:03:07 UTC