Re: WGLC: p1 MUSTs

Continuing on #484 ...

On Apr 30, 2013, at 11:54 AM, Alex Rousskov wrote:

>> Senders MUST exclude the userinfo subcomponent (and its "@"
>> delimiter) when an "http" URI is transmitted within a message as a
>> request target or header field value.
> 
> The above MUST should not apply to proxies, right? Policing forwarded
> URLs will break applications and policing forwarded extension header
> fields is not possible at all and would violate the "MUST forward" rule.
> How about "senders MUST NOT generate"?

Right, fixed.

>> A server MUST be prepared to receive URIs of unbounded length

> This MUST may be demoted to "ought" because "be prepared" is too vague
> (but see below for a related missing MUST).

Fixed.

>> A server MUST be prepared to receive URIs of unbounded length and
>> respond with the 414
> 
> Please insert a second MUST after "and": "and MUST respond".

Fixed.

>> Multiple header fields with the same field name can be combined into
>> one "field-name: field-value" pair
> 
> Should this be a MAY as in "The recipient MAY combine multiple ...". As
> worded now, it is not clear whether a proxy is allowed to combine
> headers when forwarding them. Note that this affects extension and other
> headers that a proxy may not understand (but may still want to combine
> if allowed to do so).

Fixed.

> A server MUST be prepared to receive request header fields of
>> unbounded length and respond
> 
> Consider removing the above MUST but please add MUST after "and": A
> server ought to be prepared to receive ... and MUST respond ...
> See above for discussion of a similar MUST that applies to URIs of
> unbounded length.

Fixed.

>> A client MUST be prepared to receive response header fields of unbounded length.
> 
> Same here, except no new MUST is needed.

Fixed.

>> If chunked is applied to a payload body, the sender MUST NOT apply
>> chunked more than once
> 
> The precondition is bogus: If chunked is NOT [yet?] applied to a payload
> body, the sender still MUST NOT apply chunked more than once!

Fixed.

>> the sender MUST NOT apply chunked more than once
> 
> This needs to be rephrased to make it clear that proxies are not
> responsible for dechunking multiple chunked encodings to make the
> forwarded message comply with this MUST. For example, we could say: "the
> sender MUST NOT generate messages with multiple chunked encodings".
> 
> Please note that both the proposed "multiple chunked encodings" and the
> existing "more than once" wordings imply that foo,chunked,bar,chunked
> combination is also not allowed.

Both are intentional.  A proxy is required to fix framing problems
in received messages (or reject them without forwarding).  Such
messages might be crafted to bypass security filters.

>> A server MUST send an empty trailer with the chunked transfer coding
>> unless at least one of the following is true:
> 
> This should be relaxed to "A server MUST generate ..." because a proxy,
> in general, does not know whether bullet #2 ("the trailer fields consist
> entirely of optional metadata...") is true. Even though chunking is a
> hop-by-hop mechanism, proxies ought to forward Trailers whenever
> possible, right?

Fixed.

>> a client MUST send only the absolute path and query components of the
>> target URI as the request-target
> 
>> To allow for transition to the absolute-form for all requests in some
>> future version of HTTP, HTTP/1.1 servers MUST accept the
>> absolute-form in requests
> 
> Should the first "MUST send" be relaxed to "MUST generate" so that the
> proxies do not block the apparently anticipated "transition to the
> absolute-form for all requests" by stripping URIs as they forward them?

No, since then such proxies won't interoperate with older servers.
I wish we could just delete the second requirement, since it has
been proven to be unreliable in practice and wouldn't be reliable
for any HTTP/1.x (it isn't needed for support of HTTP/2).

>> In order to avoid request loops, a proxy that forwards requests to 
>> other proxies MUST be able to recognize and exclude all of its own
>> server names
> 
> Several intermingled issues here:
> 
> 1) The "other proxies" prerequisite is a red herring IMO. Any proxy
> should avoid request loops. If a proxy that is not configured to forward
> request to other proxies sends an "origin server" request to itself,
> such a request may still create a loop.

Well, if it sends an origin server request to itself, then as
the recipient it takes on the role of an origin server or gateway,
not a proxy, and the requirement (as stated) no longer applies ...

> I think the following would be
> better: "In order to avoid request loops, a proxy MUST ..."

Since this section is about forwarding, I have changed it to

   An intermediary MUST NOT forward a message to itself unless it is
   protected from an infinite request loop. In general, an intermediary ought
   to recognize its own server names, including any aliases, local variations,
   or literal IP addresses, and respond to such requests directly.

> 2) If we want the proxy to recognize and exclude, let's demand that
> instead of just demanding that the proxy is able to do that (but
> possibly does not do it): "a proxy ... MUST recognize and exclude ...".

Fixed.

> 3) This MUST may benefit from some polishing to clarify what "exclude"
> means. I think it means "reject" in this context.

Not necessarily.

> 4) The "recognize" part can be dropped because recognition is implied by
> the "exclude" requirement.
> 
> At the end, we may arrive at something like this:
> 
> "In order to avoid request loops, a proxy MUST reject requests for
> itself, including requests where the server address is formed using a
> proxy domain name, its aliases, local variations, or literal IP addresses."

It can handle the requests itself.  I have rephrased it.

>> A client that does not support persistent connections MUST send the
>> "close" connection option in every request message.
> 
> Including a CONNECT request message?

Yes (they are orthogonal).

> A client that pipelines requests MUST be prepared to retry those requests
> 
> MUST be prepared to retry but does not have to retry? Or MUST retry?

Changed to "SHOULD retry unanswered requests".

> A client that pipelines requests MUST be prepared to retry those
>> requests if the connection closes before it receives all of the
>> corresponding responses.
> 
> Please clarify that the client MUST retry unanswered requests and not
> all "those requests" it pipelined.

Fixed.

> MUST NOT pipeline on a retry connection until it knows the connection
>> is persistent.
> 
> Is it really possible to know that a connection _is_ persistent?

Fixed, and rephrased to note that it is due to the TCP reset problem.

...

> And here is a list of MUST-level requirements that are missing an
> explicit actor on which the requirement is placed. Most of these should
> be easy to rephrase to place the requirement on the intended actor
> (e.g., "A proxy MUST" instead of "header field MUST":
> 
>> An unrecognized header field received by a proxy MUST be forwarded
>> downstream

Fixed (and then removed as it was a redundant copy).

>> The host MUST NOT be empty; if an "http" URI is received with an
>> empty host, then it MUST be rejected as invalid.

Fixed.

>> the TCP connection MUST be secured,

Fixed.

>> These special characters MUST be in a quoted string

Removed as it was redundant to ABNF.

>> the message framing is invalid and MUST be treated as an error

Fixed.

>> a response message received by a user agent, it MUST be treated as an
>> error

Fixed.

>> The trailer MUST NOT contain fields

Fixed.

>> the Host field-value MUST be identical

Fixed.

>> the Host header field MUST be sent with an empty field-value.

Fixed.

>> The "Via" header field MUST be sent by a proxy

Fixed already for another ticket.

>> the connection MUST be closed after the current request/response is
>> complete

Removed as redundant.

>> all messages on a connection MUST have a self-defined message length

Changed to "need to".

>> the first action after changing the protocol MUST be a response

Rephrased.


> Please be careful with "send" and "generate" when fixing the above
> actorless rules so that the proxies do not accidentally become
> responsible for policing traffic where unnecessary.
> 
> Thank you,
> 
> Alex.

Committed in

  http://trac.tools.ietf.org/wg/httpbis/trac/changeset/2334

Thanks for the detailed review,

....Roy

Received on Thursday, 1 August 2013 10:46:03 UTC