Re: Limits on number of interim responses in HTTP clients/proxies

> On Jan 16, 2024, at 2:05 AM, Marius Kleidl <marius@transloadit.com> wrote:
> 
> Dear working group,
> 
> the HTTP client in Go's standard library includes an (undocumented, as far as I know) limit on how many interim or final responses it will receive for a single requests. In general, the client handles interim responses well, but if more than 5 responses are received, the entire connection/stream is aborted. This also results in the final response being lost. More details on this hard-coded limit can be found at https://github.com/golang/go/issues/65035.
> 
> While this is a limitation in Go's client and should be addressed there, it does raise two questions:
> 1. Is there a recommendation in any RFC to limit the number of received responses to avoid a certain type of attack or abuse pattern? I was not able to find such in RFC 9110 or 9112.

No, because there is no such attack. The client can close at any time, if it wants, but multiple interim responses can sustain a long-lived connection while the server generates a response. For example, the client might have requested something that takes a long time to generate, might be responding with upload checkpoint metadata, or might be using the HTTP connection for chat and the final non-interim response would close that long-lived connection. These are less common now due to the presence of websockets and javascript-heavy clients, but they are still present in the protocol.

> 2. Do people know whether other clients or proxies implement a similar limit? Proxies should not interfere with interim responses AFAIK, but the reality might be different. It would be helpful to know if this limitation is common.
> 
> A bit more on the background: As part of draft-ietf-httpbis-resumable-upload, we are considering allowing an endpoint to regularly send interim responses to update the client on the processing progress. Of course, this feature is contra-productive if the client aborts the connection after too many responses. In the past, HTTP endpoints usually didn't send many interim responses, so such limitations were not causing regular problems.
> 
> I would be happy to hear if someone has more insights into this.
> 
> Best regards
> Marius Kleidl

I have never heard of such a limit before. It's a bug in their implementation.

....Roy