Re: #879: Should servers interpret Transfer-Encoding in 1.0 requests?

On Mon, Jul 19, 2021 at 03:48:45PM +1000, Mark Nottingham wrote:
> So it sounds like 'MUST be handled as an error' might work (perhaps with some
> additional details about why it's unsafe): a specific compatibility mode that
> doesn't claim conformance is out of scope. We might want to add something
> like:
> 
> > Note that some implementations can be configured to process messages with
> > both Content-Length and Transfer-Encoding to improve compatibility with
> > non-conformant peers. Such modes should be only used when the security and
> > interoperability considerations (as per above) are understood.

After giving it more thoughts, I'm not sure this is either necessary nor
sufficient. The problem mostly is about 1.0 intermediaries forwarding T-E
delimited messages to 1.1-capable servers without understanding T-E
boundaries. By definition we cannot act on those outdated intermediaries
and we cannot completely rule out the risk that some of them would simply
skip a chunk-size as an invalid method (e.g. "0<CR><LF>") or consume the
request body as defined by the C-L header and strip it before passing
the request, or even confuse a trailing chunk size with a valid request.
In such a case, a second request made of a method consisting only in hex
digits could be accepted by the intermediary, and be used as a chunk size
by the net hop or final server (especially if the URI is made of a ';'
which can be taken for a chunk-extension).

E.g.

     GET /req1 HTTP/1.0\r\n
     Transfer-encoding: chunked\r\n
     Connection: keep-alive\r\n

     A0 ;/req2 HTTP/1.0\r\n
     Connection: keep-alive\r\n
     X-folded-header: ~130 bytes of garbage\r\n
      0\r\n

     GET /req3 HTTP/1.0
     Connection: keep-alive

A lenient 1.0 proxy could take "A0" for a valid method, and ";/req2"
as a valid URI, and pass this on. The 1.1 server waiting for a chunk
size would read A0 as a 160 bytes chunk, followed by a chunk extension,
then some contents, till the " 0\r\n" that completes a folding header
field for the proxy and that could validate the final chunk for the
1.1 server. Then we're desynchronized.

For this to happen it would require that the proxy supports pipelining,
or that it's very lenient (like those which used to work on packets).
But again we're speaking about an uncontrolled, outdated 1.0 proxy.

As such, I'm wondering now if the safest solution that doesn't risk
breaking tons of stuff wouldn't simply be to state that an 1.0
request that carries a Transfer-Encoding header field must always be
considered as the last one on that connection and that the server must
close after responding to it. In the worst case the server will parse
more data than expected for what it will take for a single request
while the proxy would take it for two or more, but this will prevent
such a connection from being desynchronized, cached from being abused
or filtering from being bypassed. And it will implicitly address the
case of C-L + T-E in 1.0 through these.

Note, we could also state that C-L + T-E indicates that the sender is
not fully 1.1-compliant, thus probably at risk, and apply the same
fall back there.

It would then result in something like this:

  - a request received with both Content-Length and Transfer-Encoding
    MAY be rejected as invalid, and in any case the connection MUST
    always be closed after processing that request. Such a request
    indicates that the sender does not fully respect the standard
    and might be at risk of conveying a request smuggling attack ;

  - an HTTP/1.0 request received with a Transfer-Encoding header field
    MAY be rejected as invalid, and in any case the connection MUST
    always be closed after processing that request. Its sender might
    not be aware of the chunked transfer coding and might be at risk
    of conveying a request smuggling attack.

I don't think it would be too difficult to enforce for most
implementations and shouldn't visibly affect any legitimate traffic.

Thoughts ?

Willy

Received on Monday, 19 July 2021 08:49:51 UTC