Re: clarify some MUST requirements in HTTPbis part 1 section 3.3

On 12/02/2011 03:02 PM, Willy Tarreau wrote:
> On Fri, Dec 02, 2011 at 02:12:14PM -0700, Alex Rousskov wrote:
>>> Actually, it's even more common to see this :
>>>
>>>     5) ignore 90% of headers you don't care about and for which you
>>>        don't know how to determine the validity, and apply 4) when
>>>        you care about the header.
>>
>> Actually, RFC 2616 does define what a valid header is. Any header not
>> explicitly defined in that RFC still has to obey the rules for extension
>> headers. The proxy does not have to follow any other rules to stay
>> compliant.
> 
> Some trickier situations have become apparent since 2616. One of them
> being the multiple Content-Length. Each of them taken individually is
> OK. Having them together is wrong, still it happens and causes folding,
> resulting in "100, 100" appearing in the value.

I am aware of these issues, but I do not see why they make the situation
we are discussing any "trickier". Multiple Content-Length header fields
are not valid per RFC 2616 rules. So are multiple Content-Length header
field values. And so is folding of Content-Length headers. All of that
is already covered by RFC 2616.

What is not fully covered, is what the proxy (and other agents) are
supposed to do when they receive invalid values. That is what HTTPbis is
trying to address. The current HTTPbis text is not clear when it comes
to proxy reaction to what HTTPbis considers "fixable" or "correctly
interpretable" header values.


>>> I don't see how we can impose 2) for the 90% headers that a proxy
>>> does not understand and just ignores.
>>
>> Per my proposed wording, those headers do not need fixing: Any extension
>> header can be forwarded "as is" while staying compliant. There are some
>> nuances related to handling of multiple extension headers with same
>> names, but the same "as is" rule works for those as well.
> 
> It's not that black and white. For instance in haproxy I don't use the
> Host header, I leave it unaltered so that the next hop gets the real
> value. So if there are crappy values there, they will pass through.
> It is possible that in your proxy you heavily rely on Host and that
> you find it obvious to fix it.

Please note that you seem to divide headers into "the ones the proxy
understands" and "the ones the proxy does not understand". Since such
classification is beyond HTTP specification boundaries, it does not help
and, indeed, it creates a lot of gray areas on its own.

I divide headers into "violating HTTP specs" and "not violating HTTP specs".

Now back to your example with the Host header, which I find very
valuable for this discussion!

An HTTP proxy that talks to another proxy does not need to interpret or
understand the Host header value. Can we require that proxy to validate
the Host header against HTTP specs? I do not think so. While such
validation would certainly be possible, it would be a waste of resources
in most cases. Thus, the "proxy must not forward invalid headers" is not
a good rule! Keeping the classification black and white helped us reach
that conclusion.

I believe there are many similar cases. For example, it would be wrong
to require all proxies to validate Date headers.

Are there headers that any HTTP proxy has to validate? Yes, correctly
handling all headers that deal with message framing is required for any
HTTP proxy to work correctly. Which of those headers are end-to-end and,
hence, can be "forwarded"? I can think of only the Content-Length header
but please correct me if I there are others.

Given the above, we can restrict the rule to:

  When forwarding an upstream message with an invalid Content-Length
  header, a proxy MUST NOT send a message with an invalid
  Content-Length header to the next hop.

followed by the current HTTPbis rules about appropriate reactions to
some specific kinds of malformed Content-Length headers.



>> If some specification defines an extension HTTP header that requires
>> special forwarding beyond RFC 2616 rules, that specification itself is
>> broken.
> 
> It's not that simple. There are issues which only happen when forwarding.
> For instance, I once used to add "Connection: x-forwarded-for" to my
> requests to some reverse-proxies to show that my IP address was not
> passed to the origin server. This is a typical example of something
> which only affects forwarding. X-Forwarded-For is not part of a spec
> but you can do that with other headers and observe surprizing results.

Sorry, I do not see how your example contradicts what I said. Correct
handling of the Connection header is specified in RFC 2616. Connection
header value may include extension headers so adding X-Forwarded-For to
Connection is fine as far as RFC 2616 is concerned. A compliant proxy
receiving such a Connection header would have to remove the
X-Forwarded-For header from the received request, even if it knows
nothing about that particular extension.


Cheers,

Alex.

Received on Friday, 2 December 2011 23:14:09 UTC