Re: Of HTTP/1.1 persistent connections and TCP Keepalive timers

On Thu, 2 Nov 2000, James Lacey wrote:

> Jeff.Hodges@kingsmountain.com wrote:
> 
> > I'm curious about how HTTP/1.1 [RFC2616] persistent connections typically work
> > with respect to the typical browsers out in the wild today (Netscape &
> > Microsoft being the two I'm particularly interested in). If I cause a browser
> > to send a GET request for a given URL (using HTTP/1.1) to a server, and the
> > server doesn't encounter any errors in processing it and responding, and then
> > I (say) don't touch the browser for hours, what *typically* happens to the
> > established HTTP/1.1 (-over-TCP) connection?
> 
> The web server will close the connection due to inactivity.
> 
> You have to realize that web servers are trying to service literally
> thousands of clients with only a relatively few TCP/IP connections.
> If you don't use it you lose it.

It is fair to say that the client has no idea of what sort of resource 
limits the server may have, so it should leave the connection open 
unless it has some particular reason (eg. wanting to limit the total
number of connections it has open to all servers) to close it.  In 
reality, common browsers probably put some fixed time limit on how long
they keep a connection open; if anyone wants to know exactly what 
whatever browsers they are concerned with do, try it.

On the same token, the server has no great desire to arbitrarily close 
connections, so it will leave it open unless it has some "reason" to 
close it.  Currently, many servers use a very simple metric for this,
which is a fixed timeout of x seconds after the last response was 
sent.  This is arguably a very poor metric to use, there has been some 
small amount of research on adaptive keepalive timeouts based on load.
The server, however, has a lot more information on which to base a decision
to close a connection.

It isn't that this sort of behaviour doesn't follow the spec, but simply
that there is no need to embed this sort of runtime information in a
protocol spec.  If you read the spec carefully, you will note it goes to
great lengths to ensure that the server may close the connection whenever
it is idle, which requires things like a half-duplex close, etc.

The bottom line, however, is that if you open a connection and make 
one request every few hours, I really don't see why you should care
if it is persistent or not, since persistent connections are just an
optimization.

> Also, most web servers actively look for reasons to close a connection
> to a client. For example, if the web server generates any dynamic content
> for the client, then it will usually close the connection after the response it
> sent back. Regardless of whether or not the client supplied a Connection:
> Keep-Alive header or not. The reasoning behind this is that if the sever
> had to generate dynamic content on your behalf, then you've had your share
> and its time to give some other poor slob a turn.

Huh?  That would be a webserver with some very... odd ideas.

If you are talking about "Connection: Keep-Alive" then you appear to be
talking about HTTP/1.0, in which there is no chunked encoding so
unless the server puts a content-length on its dynamic content
(which is perfectly possible for it to do, but many don't for
reasons that are also perfectly legitimate) then there is no way
to use a persistent connection since in that case the only
end-of-reponse marker you have is the close of connection.

> 
> >
> >
> > I note that RFC2616 says (in part)..
> >
> >                              :
> > 8 Connections
> >
> > 8.1 Persistent Connections
> >                              :
> >    HTTP implementations SHOULD implement persistent connections.
> >                              :
> >    A significant difference between HTTP/1.1 and earlier versions of
> >    HTTP is that persistent connections are the default behavior of any
> >    HTTP connection. That is, unless otherwise indicated, the client
> >    SHOULD assume that the server will maintain a persistent connection,
> >    even after error responses from the server.
> >                              :
> >
> > As it is written, this effectively puts the responsibility for closing the
> > HTTP/1.1-cum-TCP connection on the client.
> 
> Nope. See my comments above.
> 
> Also, you have to realize that just cuz  the client sends a
> Connection: Keep-Alive header it is in no way a guarantee
> that the server will not close the connection after the
> response is sent back.
> 
> There is a HUGE difference between the way the HTTP spec
> is written and the way that web server's are actually designed
> to work on the internet.

The HTTP spec also doesn't describe what sort of config files you
should use for your server, since that isn't part of the protocol either.

> 
> I'm not saying that the web server designers violated the
> HTTP protocol. Rather they have simply done what they
> have to do in order to protect their web server from
> attacks, deadlocks, and starving clients.
> 
> Read further in the spec and you will see that the HTTP
> spec says that unsafe methods should not be pipelined.
> 
> An unsafe method is a method that in some sense changes
> the state of the server and will not necessarily generate the
> same response every time it is executed.
> 
> On the internet unsafe methods are typically used to
> represent a clients actions on the internet (i.e. I've just
> sent a request to buy product X with my credit card
> number aaaa-bbbb-cccc-dddd). Before submitting
> another unsafe method I should be allowed to get
> feedback about the current unsafe method and determine
> if I wish to proceed with the next unsafe method or not.
> So, when receiving an unsafe method (POST) most
> web servers will close the connection after the response
> is generated. Even if more unsafe methods have been sent
> into the pipe. They are simply discarded.

Umh... again, your reasoning here is a little confused.  Closing the 
connection after sending the response to a POST by no means ensures that
this problem is avoided and, at that state of the game, is pointless.

It is legitimate to send a non-idempotent request with several idempotent
requests pipelined after it.

I think you are again getting confused by the requirement for a
content-length or chunking or the lack of a response body in order to
do persistent connections.

Received on Thursday, 2 November 2000 09:24:52 UTC