Re: Fwd: Re: [tcpm] FW: Call for Adoption: TCP Tuning for HTTP

On Wed, Mar 02, 2016 at 03:34:42PM -0800, Joe Touch wrote:
> 
> 
> On 3/2/2016 3:21 PM, Willy Tarreau wrote:
> > On Wed, Mar 02, 2016 at 02:34:38PM -0800, Joe Touch wrote:
> >> - it has significant errors
> >>
> >> 	TIME-WAIT issues apply to servers, not clients.
> > 
> > Sorry but no it's the opposite. 
> 
> TIME-WAIT is a state caused by the side that closes the connection.

... that closes the connection *first* (since both sides close it).
This point is important because it means some proxies often should
better wait for a passive close from a server than deciding to
close themselves.

> In the bulk of HTTP connections, the server closes the connection,
> either to drop a persistent connection or to indicate "EOF" for a transfer.

Yes.

> Clients generally don't enter TIME-WAIT, so reducing the time they spend
> in a state they don't enter has no effect.

They can if they close first and that's exactly the problem we absolutely
want to avoid. There are certain cases where we had to put warnings in
rfc7230/7540, especially in relation with proxies. The typical case is
when a client closes a connection to a proxy (eg: a CONNECT tunnel) and
the proxy is supposed to in turn close the connection to the server. In
this case the proxy is the connection initiator, and it can very quickly
condemn all of its source ports by accumulating TIME_WAITs there. But the
same problem exists with idle persistent connections that clients must
avoid to close themselves if there's any hope the server will close soon.

> > A server has no issue with knowing that
> > a SYN belongs to a new session by seeing its ISN greater than the end
> > of the previous window. 
> 
> That's exactly the reason the server keeps information in the TIME-WAIT
> state.
> 
> > On the opposite, a client cannot know if the
> > remote server it wants to connect to is safe for reuse 
> 
> TIME-WAIT isn't just for new connections; it's to protect against
> injecting traffic from previous connections that is delayed into new
> connections...

Yes I'm well aware of this :-)

> > and will refrain
> > from establishing a new connection during the whole TIME_WAIT state,
> > effectively preventing itself from doing its job.
> 
> If that's what it doesn, that's not TIME-WAIT - it's some new state in
> the OS to avoid the possibility of hitting a TIME-WAIT at the server.
> That's mislabeled at best, and defeats the entire purpose of the
> TIME-WAIT at the server anyway.

No I'm not saying any such thing, I'm saying that by all means the
server must close first to keep the TIME_WAIT on its side and never
on the client side. A TIME_WAIT on a server is very cheap (a few tens
of bytes of memory at worst) and can be recycled when a new valid SYN
arrives. A TIME_WAIT on the client is not recyclable. That's why
TIME_WAIT is a problem for the client and not for the server.

The problem is that in some cases it's suggested that the client
closes first and this causes such problems. The only workaround for
the client is to close with an RST by disabling lingering, but that's
really ugly and unreliable : if the RST is lost while the server is
in LAST_ACK (and chances are that it will happen if the ACK was lost
already), the new connection will not open until this connection
expires.

So by all means we must do whatever we can to avoid to cause
TIME_WAITs to be accumulated on the client side and that was the
point mentionned in the document since it's supposed to be used as
a reference for future protocol designs. For example in HTTP/2, the
GOAWAY frame makes this a bit easier to take care of, since we can
declare an intent to close that will cause the other side to close.

Also, there are people who face this issue and work around them using
some OS-specific tunables which allow to blindly recycle some of these
connections and these people don't understand the impacts of doing so.
The doc will have to be clear enough to discourage them from doing so,
and to adapt the client code instead.

Willy

Received on Thursday, 3 March 2016 06:56:22 UTC