Re: HEADERS and flow control

On 2014–05–19, at 10:50 PM, Greg Wilkins <gregw@intalio.com> wrote:

> If the protocol does not apply flow controlled to headers, then somebody will eventually abuse this as a way to get around flow control and start sending bulk data as headers rather than data.   Once a few start doing it, then it will become main stream.

The user has no motivation to get around flow control.

1. The server flow control window only limits uploads. To download faster, the client only needs to set their limit high. (There’s no good reason not to do so.)
2. TCP flow control is redundant with HTTP/2 flow control at the connection level. Per-client throttling is more likely to be implemented at the TCP level.

Putting these together, bypassing flow control would at worst allow a local application to defeat its host server/user-agent’s transmission regulation. Such behavior has already been proposed as a feature (“priority level zero”). The proposal was rejected, but it’s not particularly dangerous.

> This is exactly what happened with browsers switching to using 6 connections instead of 2, because some started to do it to improve their speed.  It started with individuals changing their own browsers, but then eventually one browser switched and everybody had to follow to stay competitive no matter how badly it abused the protocol!

The user (of the user-agent) cannot decide to send bulk data as headers instead of data. That is part of the application protocol. The common methods used on the Web are mostly fixed in this respect. It takes some effort to package bulk data as cookies, and it still won’t stream unless you break it into multiple header blocks, which aren’t supported by current APIs, or multiple entity-streams.

If someone wants to bypass flow control on uploads, they will do what you already described: multi-connection. But, most folks don’t have a problem with not being able to upload fast enough, and since most uploads go to some kind of content provider who has a relationship with the user, the problem is more likely to be solved without hacking.

As for downloads, multi-connection is an obvious hack, but this is a well-known strategy and countermeasures must be applied where necessary anyway.

> Other than the meta data defined by HTTP, all other headers are essentially application data and must be flow controlled.  

The application defines what should be sent in headers. The HTTP/2 spec should include a stronger warning that header blocks should be minimized or broken-up to avoid service degradation due to flow control.

> The deadlock scenario described is only a problem if the initial compressed headers are larger than the initial flow control window, which at 64kB is highly unlikely with current usage.  However, if we don't put flow control on headers, then I think we will very soon start seeing >64kB headers!

There is already a per-implementation, arbitrary maximum header set size. I think there should also be a maximum header set size, because intermediaries are obliged to use RAM to decode headers even if they aren’t added to the reference set. I hadn’t noticed that before. In any case, letting headers grow without bound is a recipe for dropped streams and no competent app developer would do it.

> I definitely support putting flow control on all non control frames within a stream.   If this means we have to do something a bit ugly to avoid the deadlock, then I think that is far better than unconstrained meta-data.

Why not flow-control the control frames too? Then HTTP/2 would be on par with TCP.

The flow control protocol is misplaced and redundant with prioritization. I suspect that it would not exist but for legacy SPDY implementations. But, aside from adding some overhead (at least 24 bytes per TCP frame, approximately doubling the overhead of TCP) and implementation complexity, it’s harmless. Fortunately, its absence is harmless too.

Received on Wednesday, 21 May 2014 07:58:08 UTC