- From: Jamie Lokier <jamie@shareable.org>
- Date: Fri, 12 Mar 2004 19:07:10 +0000
- To: Scott Lawrence <scott@skrb.org>
- Cc: ietf-http-wg@w3.org
Scott Lawrence wrote: > > I'm not entirely sure if this means _network_ errors (such as TCP > > shutdown or reset), or HTTP Error Status codes 4xx and 5xx. I guess > > it means HTTP Error Status codes, because of the implication that the > > connection might continue. > > It does mean HTTP Error Status responses (any status > 299). Thanks. Why > 299? To my mind, error status is > 399 (i.e. 4xx = Client Error, 5xx = Server Error). If redirections are Error Status responses for the purpose of low level client behaviour in section 8.2.2, that is poorly worded and should be clarified in the Errata. If I am confused, surely some client implementations will have it wrong too. That's critical: the server _has_ to know whether the client will abort sending the request body or not, to decide how to continue with the connection. > > 2. Regarding my second question, whether a non-error response can be > > safely transmitted without reading the request entity. Is it allowed? > > Does the connection have to be made non-persistent; i.e. closed afterwards? > > Not if the client says it's HTTP/1.1. If the client is 1.0, then you > have to assume a non-persistent connection by default. Assume persistence has already been determined appropriately, by looking for Connection: close and/or Connection: keep-alive in the request. After transmitting the response entity and discovering that it didn't need the request entity after all, and it wasn't an error response, it's too late for the server to change the response headers: they are transmitted already. The server will not have put Connection: close in the response header, unless that was requested by the client, or it was HTTP/1.0 without keep-alive, or some other conditions applied. So the server will transmit the response... And then it has to shut down the connection, right? (All the while reading data from the client to avoid the TCP RST problem). > > If you have enough information to begin the response just from reading > > the request headers, then you are free to send the response headers > > right away; even the 1xx response is optional - you can skip right to > > sending the final response. (Note: the rules of the protocol don't > > forbid this - what any given client will do with this is a different > > question that can be answered only with testing). > > Yes. Thanks. > > If 100 (Continue) isn't transmitted, some clients may start > > transmitting their request entities anyway. Therefore there's no way > > for the server to determine whether data received from the client is > > the request entity or the next request, if it wants to try avoiding > > 100 (Continue) to produce a non-error response without using the > > request entity. > > No, you can tell from the headers whether or not there is a request > body. If there is a non-zero Content-Length, or a 'Transfer-Encoding: > chunked', or a Content-Type specifying a multipart encoding, then that > indicates the presence of a body. If none of the above are there, the n > there is no body and the next byte starts the next request. That's not what I mean. What I'm saying is: if I _don't_ send 100 (Continue), I don't see a way to determine whether the client _chose_ to not transmit the request entity _despite_ the Content-Length or Transfer-Encoding headers. Therefore, after doing that, there's no way for the server to read another request, and it must shut down the connection. (Btw, any Transfer-Encoding indicates a request body, and "chunked" is required to be the last in the sequence of transfer-codings. If Content-Type specifies "multipart/byteranges", that indicates a body, but I see nothing in RFC 2616 which indicates a body for any other multipart encoding. Right?) But now on yet more diligent reading of RFC 2616, I have a question. This whole mail revolves around the question so I'll make it stand out: If the client sends Expect: 100-continue, and receives an HTTP/1.1 (or later) response which is not preceded by 100 (Continue), is the client allowed to send more requests on that connection _without_ transmitting the request entity? > > Or can it avoid reading the request entity completely, by not sending > > 100 (Continue) and somehow determining when the next request arrives > > on that connection. Is that possible? > > No, but even if it were it wouldn't save you anything - TCP is a stream; > there's no way to discard the bytes other than at the receiver (without > closing the connection, that is). Eh? If you don't send 100 (Continue) then it does save you quite a lot: the client might not transmit the request entity at all! Also, if you stop reading at the receive, eventually the TCP window fills and the transmitter stops, which reduces data transmitted overall if not all of the request entity is read. That may or may not cause deadlock, depending on the transmitter's logic. We can avoid deadlock while reading the minimum at the server, by reading only when the data is needed, writing response as we generate it, and if writing blocks, then reading until writing is unblocked, buffering the read data. This is the logic I have now. Assume in every case that the connection is persistent: the request doesn't have Connection: close, and it's HTTP/1.1. 1. If the server sends a non-error response, without 100 (Continue) and without observing any request entity prior to sending the response, then it must not try to read another request because of the ambiguity as to whether the client actually sent the request entity or not. Is this correct? 2. If the server sends an error response, without 100 (Continue) and without observing any request entity prior to sending the response, then it must not try to read another request for the same reason. Is this correct? 3. If the server sends a *non-error* response, after 100 (Continue), and before reading all of the request entity, it can assume the client will send all of the request entity or abort the connection. In this case if the server doesn't want to read all of the request entity, because it knows the entity is large and there is no point transferring the remaining data, then after it has completely sent the response entity it should signal connection shutdown (i.e. TCP FIN, shutdown(SHUT_WR)), and continue reading for a while to prevent the TCP RST problem. If the server does want to read all the request entity after it has started sending the respones, it can just do that and it should work. 4. If the server sends an *error* response, after 100 (Continue), and before reading all of the request entity, it must assume the client MAY prematurely abort transmitting the request entity either by closing the connection or transmitting a zero length chunk. In this case if the server doesn't want to read all of the request entity, because it knows the entity is large and there is no point transferring the remaining data, then it doesn't have to shutdown the connection. The client may terminate the request entity with a zero length chunk allowing the connection to continue. However, the client might not, so if there's a lot of data still coming in, the server may choose to shutdown the connection, in the same way as for 3. above, a short time after sending the error response and failing to see the client prematurely terminate the request entity. If the server does want to read all the request entity after it has started sending the response, it cannot. It has to _delay_ sending any of the response until all of the request entity has been read. 5. Because of the different required behaviours in 3. and 4., due to RFC 2616 section 8.2.2, the server and client must agree on what consitutes an "Error Status". Clearly 4xx and 5xx codes are; 1xx and 2xx are not. It isn't clear whether 3xx codes are errors. If there is disagreement either way, there are problems. If the client receives a 3xx and treats that like a non-error, it will continue transmitting the request entity. For the server to terminate that transmission, to save bandwidth and inform the client that it doesn't need all the response, it needs to shutdown the connection after the response is transmitted, in this case. If the client receives a 2xx and reacts in the same way as for an error status, it may prematurely terminate the request entity by sending a zero length chunk, or it may close the connection which, although always allowed perhaps due to user abort, is not expected for non-aborted non-error requests. Therefore for the server to not cause premature request entity termination, it must _delay_ sending the response until it has read the whole request, in this case. So if there is any disagreement among implementations over what is an Error Status for the purpose of section 8.2.2, this creates a third category of responses that the server must treat differently from 3. and 4. I see this category containing 3xx. If one of these responses is generated within the server, and the server wants to read all of the request entity eventually, then it must _delay_ sending any of the response until all of the request has been read. However if the server does not want to read all of the request entity, it cannot assume the client is likely to prematurely terminate the request entity by closing or sending a zero length chunk. So the server's behaviour with this category of responses is different from its behaviour for responses which are consistently treated by implementations as errors or non-errors. It has to use this third, more conservative strategy. 6. If the server transmits a response, and in so doing discovers that it doesn't need any of the request entity, then it can avoid sending 100 (Continue). However, if it does not send 100 (Continue) is it true that it mustn't try to read another request on that connection? I'm not sure. If the server discovers it doesn't need any of the request entity, but it would like the keep the connection persistent, it needs to use a different strategy for non-error and error responses: a. For non-error responses, if the request entity is known to be small from the Content-Length header or otherwise, it may be worth the server sending 100 (Continue) prior to the response anyway, reading the whole request entity and discarding it, and continuing to read another request. b. For non-error responses, if the request entity is known to be large, or the server doesn't wish to encourage an unknown size transmission over the network from the client, the server will avoid sending 100 (Continue), and instead shutdown the connection (carefully, avoiding the TCP RST problem) after sending the response. c. For error responses, the size of the request entity is not important. The client will abort sending the request entity, and make an appropriate decision on whether to continue using the connection. 100 (Continue) must be still be sent by the server if it wants to use the connection persistently after the error response. Is this correct? (I'm not sure). This translates to the following logic in server code. I would be grateful to anyone who can offer corrections to or insight into this logic. It seems more complicated than I'd expected to need for implementing lazy request entity reading for an application running on top of the server. a. In all of the steps below, where it says transmit 100 (Continue) that may be avoided if some request entity has already been received. b. While transmitting the response, if at any time the transmission is blocked for writing, the server should try to read data from the client and save it for when the server application reads it, in order to resolve deadlock with less tolerant clients and large messages that fill TCP windows. c. As soon as the server application tries to read any of request entity, transmit 100 (Continue). d. If the application generates a *non-error* status, and might need to read some of the request entity but hasn't yet, transmit 100 (Continue) before the status. e. If the application generates a *non-error* status, and has already read or tried to read some of the request entity, transmit the status immediately. f. If the application generates a *non-error* status, and is already able to commit to not needing any of the request entity, but the size of the request entity is believed by the server to be sufficiently small that it's worth reading in order to maintain a persistent connection, and the connection is still eligible for persistence (neither side has transmitted Connection: close), transmit 100 (Continue) before the status. g. If the application generates a *non-error* status and is already able to commit to not needing any of the request entity, and the request entity is not believed by the server to be small enough to read in order to maintain a persistent connection, transmit the status without 100 (Continue). h. If the application has already generated a *non-error* status and is subsequently able to commit to not needing any more of the request entity than it has already read (which may be none), there is no need for the server to continue reading from the client except as noted for deadlock avoidance (b.), or if the server is going to read the whole request entity anyway (f.). i. If the application finishes handling a request and it generated a *non-error* status, and did not read all of the request entity, and the connection is still eligible for persistence (neither side has transmitted Connection: close), and the server believes that the amount of request entity that has not yet been received is small enough to be worth receiving in order to maintain a persistent connection, then it should read the remaining request entity and continue using the connection. j. If the application finishes handling a request and it generated a *non-error* status, and did not read all of the request entity, and the server believes that it is not worth receiving the remainder of the request entity, it should politely shutdown the connection, with a "lingering close" to avoid TCP RST problems. Deadlock avoidance is required in conjunction with the lingering close as with (b.), by continuing to read incoming request data rather than letting the TCP receive window fill. k. If the application generates an *error* status, and might need to read some of the request entity but hasn't yet, *delay* transmitting the status until the application has completed handling this request, or can otherwise commit to not needing the request entity, or does actually read the entity and the *whole* entity has been received, or at least as much as the application will need of it. l. If the application generates an *error* status, and has already read or tried to read some of the request entity, *delay* transmitting the status until the *whole* entity has been received, or at least as much as the application will need of it. m. If the application generates an *error* status, and is already able to commit to not needing any of the request entity, transmit the status immediately. The size of the request entity is not important, because the client is expected to prematurely abort a long one. n. If the application has already generated an *error* status and is subsequently able to commit to not needing any more of the request entity than it has already read (which may be none), there is no need for the server to continue reading from the client except as noted for deadlock avoidance (b.). However, if the connection is still eligible for persistence, the server should continue reading for a limited time, number of bytes or other policy determined by the server (see o.), speculating that the connection will be used for another request. o. If the application finishes handling a request and it generated an *error* status, and did not read all of the request entity, and the connection is still eligible for persistence (neither side has transmitted Connection: close), the server should continue to read and discard the request entity. The client should prematurely terminate the request entity, although it might not. Therefore the server should read and discard only for a limited time, number of bytes, or other policy determined by the server, and if the end of the request is not seen by then it should politely shutdown the connection, with a "lingering close", as in (j.). p. If the application generates a status which it's thought some clients may treat as an error and some as non-error for the purpose of RFC 2616 section 8.2.2, a conservative mix of the above rules is required. 3xx status codes might be in this category. -- Jamie
Received on Friday, 12 March 2004 14:07:13 UTC