Re: Question on flow control for a single file transfer

Some (possibly well-known) comments on TCP flow control - I guess HTTP/2 could run into similar issues and solutions as well...

* Many TCP stacks have complicated heuristics to determine the TCP receive window they advertise to the sender. In addition to free receiver buffer space, other metrics such the estimated path BDP, the socket/application read rate, etc. may be taken into account as well.

* In the worst case, the buffer space needed in a receiver is larger (by factor 2) than what is actually advertised to the sender, if out-of-order data should be stored. In case of TCP, out-of-order data can also be discarded, but most modern stacks buffer such data for efficiency, AFAIK.

* In TCP, it is also not uncommon to oversubscribe memory, i.e., if there are many TCP connections, the sum of all advertised TCP receive windows may exceed the memory that is actually available. This heuristic assumes that not all connections need buffer at the same time. If this assumption turns out to be wrong, the receiver can still drop incoming packets if it cannot allocate buffer for it. This is not different to network congestion, i.e., it will affect throughput, but the data transfer will continue.

* However, a TCP sender must respect the receive window advertised by the receiver. TCP receivers often check this (amongst others, for security reasons).

My 2 cents

Michael



________________________________
Von: willchan@google.com [willchan@google.com]" im Auftrag von "William Chan (陈智昌) [willchan@chromium.org]
Gesendet: Montag, 4. November 2013 04:54
An: Peter Lepeska
Cc: Yoav Nir; <ietf-http-wg@w3.org>; Martin Thomson
Betreff: Re: Question on flow control for a single file transfer

http://en.wikipedia.org/wiki/Flow_control_(data) says "In data communications, flow control is the process of managing the rate of data transmission between two nodes to prevent a fast sender from overwhelming a slow receiver."

Guesstimating BDP is only important if the receiver cares about maximizing throughput. Which hopefully it does, but there's no guarantee. Sometimes due to resource constraints, the receiver cannot accept that much data, and it asserts flow control in this case. And senders *need* to respect that. Otherwise a receiver with any sense, like a highly scalable server, will terminate the connection since the peer is misbehaving.


On Sun, Nov 3, 2013 at 7:44 PM, Peter Lepeska <bizzbyster@gmail.com<mailto:bizzbyster@gmail.com>> wrote:
Sloppiness? I don't get that. The sender's job is to transmit the data as fast as possible, not to respect the receiver's best guesstimate of available bandwidth sent ½ RTT ago. In this case, the sender's job is to keep the TCP buffer full of data so it can send it when it has the opportunity to.

Respecting the peer's receive window in the single file send case is harmless at best and detrimental otherwise.

Peter

On Sunday, November 3, 2013, William Chan (陈智昌) wrote:

I don't feel comfortable encouraging such sloppiness, I worry about future interop. Respecting a peer's receive window isn't hard. Just do it :)

And even though wget doesn't support upload (to my knowledge, but I'm not an expert), a command line tool may upload, in which case it should definitely respect the peer's receive window.

On Nov 3, 2013 6:22 PM, "Yoav Nir" <ynir@checkpoint.com> wrote:

On Nov 3, 2013, at 1:25 PM, William Chan (陈智昌) <willchan@chromium.org> wrote:


It's probably understood already, but just to be clear, this is receiver controlled and directional. Unless you control both endpoints, you must implement flow control in order to respect the peer's receive windows, even if you disable your own receive windows. Cheers.


This discussion started with tools like WGET. If all you're ever sending is one single request equivalent to "GET xxx", you're likely fine not considering server receive window.

For a single file, the data that the client sends to the server never exceeds the default server receive window.