Comments on Section 6.1 (Persistent Connections) of HTTPbis Part 1, version 17

We are currently working on developing a model of HTTP's operation over TCP based on Part 1 of HTTPbis. This has led us to make the following 5 comments on section 6.1 of draft 17. We have also included possible replacement text to help clarify section 6.1 of Part 1, where we thought it may be useful. The final comment may touch on design, and we are not confident about our assumptions, but we set out some issues that we feel need to be resolved, and are willing to prepare replacement text if the issues can be resolved.

Regards

Jonathan Billington
Professor of Computer Systems Engineering
University of South Australia

Sonya Arnold
Research Student
University of South Australia

------------------
Comments on Section 6.1 (Persistent Connections) of HTTPbis Part 1, version 17, 31 October 2011
Input to: HTTPbis Working Group.
20 December 2011
J. Billington and S. Arnold

1.  Section 6.1.1 Purpose
Change Type: Editorial clarification
Rationale: This section includes a list of advantages of using persistent HTTP connections. The third and fourth points state:
"Network congestion is reduced by reducing the number of packets caused by TCP opens, and by allowing TCP sufficient time to determine the congestion state of the network."
"Latency on subsequent requests is reduced since there is no time spent in TCP's connection opening handshake."

These points concentrate on the opening phase of TCP causing delay, because of the 3 way handshake, but equally important is the graceful closing phase, which requires handshakes in both directions (FIN followed by ACK for each side), and the holding of the TIMEWAIT state, causing further delay. Because it is equally important to the closing phase, we feel it is useful to include this point in the list of advantages.

Proposed Replacement text:
Third point:
"Network congestion is reduced by eliminating the packets associated with TCP's three way handshake and graceful close procedures, and by allowing TCP sufficient time to determine the congestion state of the network."
Fourth Point:
"Latency on subsequent requests is reduced since there is no time spent in the double handshakes required for gracefully closing TCP connections and holding the TIMEWAIT state, and in TCP's connection opening handshake, when reopening the connection."

2. Section 6.1.2 Overall Operation, second paragraph

Change Type: Editorial clarification
Rationale: The second paragraph of this section states:
  " Persistent connections provide a mechanism by which a client and a
   server can signal the close of a TCP connection.  This signaling
   takes place using the Connection header field (Section 8.1).  Once a
   close has been signaled, the client MUST NOT send any more requests
  on that connection."

The last sentence contains a mandatory requirement that may not be able to be met by a client using pipelining. If the client signals the close, then it is OK. However, if the server signals the close in a response, further pipelined requests may be sent by the client until it receives the response with the close token. Hence we need to rephrase the requirement so that the client can meet it.

Proposed Replacement text:
Replace the last sentence by:
"Once a close has been signaled by the client in a request, or the close token has been received by the client in a response, the client MUST NOT send any more requests on that connection."

3. Section 6.1.2.1 Negotiation, second paragraph

Change Type: Editorial clarification
Rationale: The second paragraph of this section states:
  " An HTTP/1.1 client MAY expect a connection to remain open, but would
   decide to keep it open based on whether the response from a server
   contains a Connection header field with the connection-token close."

It is not clear what the intent of this sentence is. Does it imply that on the receipt of a response with a close token, the client will close the connection? Does it also imply that if the response did not contain the close token, then the client would keep the connection open? If this is the intent, then a clearer statement of it would help to exclude other interpretations, as the current text seems to allow the client latitude to close or not on either of these events.

Proposed Additional text if the above interpretation is the intent:
Add a sentence to clarify the intent at the end of the above quoted sentence:
"If the response does not contain the close token the client will keep the connection open, otherwise, on receipt of the close token in the response, the client will close the connection."

4. Section 6.1.2.1 Negotiation, third paragraph (second paragraph on page 42)

Change Type: Editorial clarification
Rationale: The paragraph states:
  "If either the client or the server sends the close token in the
   Connection header field, that request becomes the last one for the
   connection."

This is similar to item 2 above. The problem is knowing what "that request" is in the case when the server includes the close token and pipelining occurs, and what is intended by the phrase "becomes the last one for the connection". We expect that the following is the intent:
A. If the client receives a response with the close token, it will not send any more requests (as in item 2 above); and
B. Once the server has sent a response with the close token, it will discard any further requests it receives.

Proposed Replacement text, given that previous items 2 and 3 are accepted
Replace this sentence by:
"Once the server has sent a response with the close token, it will discard any further requests it receives."

5. Section 6.1.5 (page 46)

Change Type: Design/Editorial clarification
Rationale: The section states:
   "Senders can close the transport connection at any time.  Therefore,
   clients, servers, and proxies MUST be able to recover from
   asynchronous close events.  Client software MAY reopen the transport
   connection and retransmit the aborted sequence of requests without
   user interaction so long as the request sequence is idempotent (see
   Section 6.1.2 of [Part2]).  Non-idempotent request methods or
   sequences MUST NOT be automatically retried, although user agents MAY
   offer a human operator the choice of retrying the request(s).
   Confirmation by user-agent software with semantic understanding of
   the application MAY substitute for user confirmation.  The automatic
   retry SHOULD NOT be repeated if the second sequence of requests
   fails."

We believe it is useful to define terms associated with "sequence of requests". We think it is clear that "the second sequence of requests" refers to "the aborted sequence of requests" mentioned earlier in the paragraph. Thus we need to understand and define what an "aborted sequence of requests" is. To make a start on this we consider the following example:

We begin with the premise that all sequences are idempotent. Consider that the client has pipelined 5 requests and has received 2 responses when the transport connection is closed by the server. It seems clear at this stage that the aborted sequence comprises the remaining 3 requests (that have not received responses). Now we think the following occurs, where we have numbered the requests from 1 to 5 in the order they were sent.

The client automatically reopens the transport connection and resends request 3. (It cannot pipeline until it receives the response to request 3 - see paragraph 2 of section 6.1.2.2 Pipelining.) The following scenarios are possible:
a) If the server closes the transport connection before the response is received, then "the second sequence of requests has failed". The HTTP client closes its half of the transport connection, the remaining 3 requests are removed from the HTTP client, and the HTTP client user is informed of the failure. The connection could be reopened by the client user and the requests resent. In this case the HTTP client would consider the sending of these requests to be a new sequence, and hence retry them if the server prematurely closed.
b) If the response is received without the close token, then the client can resume pipelining, and could send requests 4 and 5 (and further requests). Let's say it sends requests 4, 5 and 6. This is a new sequence of requests (no longer [3,4,5]). Now if the server closes the connection before sending a response, then the automatic retry is allowed because this is now a new sequence, rather than an aborted sequence (or second sequence), even though it includes requests 4 and 5 that have been sent twice. Further, if only requests 4 and 5 were sent and the server closed, it would still be a new sequence [4,5],  rather than [3,4,5], and hence the retry would be invoked. Is this the intended meaning of section 6.1.5?
c) If the response is received with the close token, then the connection is no longer persistent and must be closed without any further requests being sent. In this case, request 4 will be sent over a new connection, initially without pipelining to align with the spirit of the approach expressed in paragraph 2 of section 6.1.2.2. If the HTTP client receives a response without the close token, then pipelining can recommence, starting with request 5. If the response does include the close token, then request 5 is treated the same way as request 4. As in a), on reopening a connection HTTP considers any sequence of requests to be new, and hence will retry if the server closes prematurely.

Proposed Additional text: We are not sure if the assumptions made above are correct or not. Once the intended meaning has been ascertained we are willing to prepare additional text to clarify the retry operation.

-------------------------------end of comments on HTTP Part 1 section 6.1--------------------------