Re: clarify some MUST requirements in HTTPbis part 1 section 3.3

On 12/07/2011 01:46 PM, Roy T. Fielding wrote:
> On Dec 7, 2011, at 10:52 AM, Alex Rousskov wrote: 
>> On 12/07/2011 10:30 AM, Roy T. Fielding wrote:
>>
>>> A proxy is responsible for complying with all requirements on senders,
>>> clients, and proxies.  That is how the entire protocol is written.

>> Does the above imply that all compliant proxies must _validate_ all
>> forwarded headers defined by RFC 2616, to make sure those headers do not
>> violate any of the 600+ MUSTs?

> In some respects, yes, such as for framing and folding.  In cases where
> the proxy is not supposed to muck with the header field, the requirements
> should be more tightly scoped.


Glad we agree that forwarded header validation and, hence, complying
with the corresponding sender MUSTs is necessary in only _some_ cases,
such as when dealing with framing, folding, and similar "core" protocol
operations.

I think I know understand that you prefer to make every sending-related
set of rules tight enough to avoid any misunderstanding on what a proxy
should do. I agree that it is possible, but I think having a set of
_default_ rules would [still] be very helpful, especially because many
HTTP-based specifications are not that tight and will not be revised as
a part of this effort.


In summary "a proxy is a sender" is the right principle, but we must be
careful when specifying sending rules because not all senders are
proxies. The proxy should not be made responsible for everything it
_forwards_, just for the "core" stuff such as framing headers. There are
several ways to polish the specs accordingly.


>> If this is how the protocol has to be interpreted, we must clarify that
>> in HTTPbis because (without an explicit confirmation) many folks would
>> continue to use a less demanding interpretation. We should then also
>> explain what a proxy should do if a to-be-forwarded header field fails
>> validation but is not needed for correct proxy operation (from UA and
>> origin server points of view)?
> 
> Why do we need to clarify that in general?  We should just fix the bugs.

I think that a few general forwarding rules are better than many
specific and often duplicated ones, but I agree that the problem can be
fixed by adding explicit forwarding rules to each protocol element
and/or rephrasing some of the existing element-specific sending rules to
not apply to proxies.

Either way, defining "[generating and] sending" differently from
"[receiving and] forwarding" would probably be necessary for the
new/fixed rules to make sense.


>> Please consider the following specific example. A proxy receives an
>> otherwise valid message with a Date header that violates the following MUST:
>>
>>  The [Date] field value MUST be sent in rfc1123-date format.
> 
> That's a bug.  For one thing, it is a poorly phrased requirement
> because it doesn't target the responsible party (in this case, the
> program that generates the field-value).

Exactly. If we just rephrase the above MUST as

    The agents MUST send Date value in rfc1123-date format

then we have suddenly required all proxies to validate forwarded Date
headers (because a proxy is an agent) which seems unnecessary and costly
in general.

If we rephrase the above MUST as

    The agents MUST generate the Date value in rfc1123-date format

then proxies are excused from validation, but the requirement itself
looks weird/overreaching: Why would HTTP care about something that is
generated unless it is actually sent?


A possible solution is to keep formulating the requirements in terms of
sending, as usual, but allow proxies to ignore sending requirements when
forwarding certain headers. There are two ways to define those "certain
headers":

1) All headers except those explicitly required to be validated (e.g,
Content-Length).

2) No headers except those explicitly allowed to be forwarded without
validation (e.g., Date).


>  For another, there is
> (or was) some other requirement somewhere that intermediaries must
> not modify so-called end-to-end metadata used for caching or that
> might appear in digest hashes.

There is actually no such explicit MUST in RFC 2616 (another bug), but
we are not talking about modification of end-to-end headers by proxies
here. We are talking about validation of received end-to-end headers by
proxies for the purpose of obeying the "proxy is a sender and the sender
MUST not send invalid stuff" principle.


>> When forwarding the message, the proxy has a few choices:
>>
>>  0) Send the Date header field as it was received.
>>  1) Do not send any Date header field.
>>  2) Create and send a new Date header.
>>  3) Reject the entire received message.
>>
>> What should a compliant proxy do?
>>
>> And Date is just one example. There are many other complex end-to-end
>> headers that a given proxy does not need to validate to function
>> correctly (from UA and origin server points of view) and that are
>> difficult or even impossible to "fix" without creating more problems.
> 
> I don't agree -- we have been fixing this type of bug all throughout
> this process.

Sorry, I am not sure what you are disagreeing with. If you are saying
that a proxy can fix invalid headers before forwarding them, then I do
not see how a proxy can reliably fix invalid Date, Server, Warning,
Vary, etc. headers (in general).


Thank you,

Alex.

Received on Wednesday, 7 December 2011 22:49:27 UTC