Re: Retry safety of HTTP requests

On 22/03/2016 6:55 p.m., Subodh Iyengar wrote:
> HTTP/1.1 and the current HTTP/2 spec don't define the concept of
> retry safety of requests, i.e. when are requests are safe to retry
> and what are the limits on how retries can be performed by the lower
> layers.


RFC 7230 section 6.3.1 titled "Retrying Requests"
<http://tools.ietf.org/html/rfc7230#section-6.3.1>

The criteria for *when* are outlined.

The *how* is not possible to be defined at the HTTP layer IMO. There are
just so many possible ways. Up to and including things like RFC 1149
Avian Carrier protocol as fallback.

So the "how" is application and indeed message specific and should be
defined at those upper layers. HTTP already provides a number of
mechanisms such as; GET, HEAD methods and/or ETag, If-*, Range headers,
etc. which can be used by the application layer to check the state
before its chosen method of retry (if wanted).


> 
> I believe this is a new item and I'm trying to gauge people's
> interest in this topic, and whether such an idea has been explored
> before in the httpwg.
> 
> I think defining retry safety will become an important problem in the
> near future. I can see several use cases if we have a proper
> definition of retry safety in HTTP:
> 
> 1) Usually an application like a piece of javascript will create a
> HTTP request and send it to a browser or mobile app which executes
> it. The browser or a mobile app might want to retry the request. In
> unreliable networks, retrying requests is important for reliability.
> The application could decide that it would create a non idempotent
> request like a POST, and have an application layer mechanism to
> enforce idempotence.

Application layer mechanisms are outside the scope of HTTP.

HTTP is not even the only protocol such messages can be sent over. There
is HTTP (1.x, 2), WebSockets, SPDY, QUICK, COAP(S),  and others all in
use various places.

> It could then tell the browser that the request
> was retry safe because it was enforcing idempotence, and let the
> browser perform optimizations to retry it depending on network
> conditions, including caching the request offline, retrying when the
> network comes back, sending the request across 2 connections on
> different interfaces, etc.

It tells the HTTP layer by using the correct RESTful method for the
semantic operation that message seeks to perform:
- a fetch (GET)
- a store (PUT)
- a removal (DELETE)
- a submission (POST)
- a state query (HEAD)
- a service capabilities query (OPTIONS, TRACE)

All HTTP software understands those semantics regardless of what other
extensions it may or may not support. Or how old it is.

> 
> 2) Some transport protocols which treat retry safe requests
> differently from non retry safe requests. For example in TLS 1.3,
> idempotent requests may be sent in a 0-RTT flight, which reduces the
> latency for the request. An application might desire a non idempotent
> request which is retry safe be sent in this 0-RTT flight.

Do not confuse transport layer frame/message with transfer layer
frame/message.

The semantics of the transfer layer (HTTP) message and even its framing
boundaries are opaque to the transport layer (TLS).

If the TLS/1.3 layer is going to perform packet retry itself in the
presence of 0-RTT flight data, then it needs to ensure that opaque data
is only ever delivered to a recipient once regardless of how many times
its actually sent on-wire. To be a "reliable transport" it needs to
guarantee that property.

I find it hard to believe that the TLS WG would allow TLS/1.3 to become
an unreliable transport layer.


> 
> 3) A non-application server such as a load balancer might also desire
> to know whether or not the client expressed that this request is
> retry safe so that it may retry this request if the backend fails
> over offering more reliability to clients.
> 
> Currently, the only semantics for an application to express retry
> safety is via idempotency (the request method). The status quo shows
> that even this is not expressive enough, since even though GETs are
> not technically idempotent, some browsers will retry them anyway.

What? thats false.

RFC 7231 section 4.2.2:
 "PUT, DELETE, and safe request methods are idempotent".

RFC 7231 section 4.2.1:
  "GET, HEAD, OPTIONS, and TRACE methods are defined to be safe".

thusly "idempotent" == PUT, DELETE, GET, HEAD, OPTIONS, TRACE

plus any other extension method defined as being idempotent or safe.


Note that certain response status codes also indicate explicitly that
nothing reached the origin. So their request is now retriable for
exactly +1 try despite its natural idempotency.

HTTP/2 extended that with GOAWAY to mark a range of pipelined request
streams.


IIRC we have a WG consensus that applications using methods counter to
their documented idempotency are not HTTP compliant. Such software is
explicitly left to face the consequences of its non-compliance when the
HTTP layer "unexpectedly" treats the message according to correct semantics.
Your application(s) treating GET as non-idempotent is a case for that.


> 
> Having an document which would define a new property like
> retry-safety would help apps take advantage of reliability features
> of browsers for requests other than GETs and also prepare HTTP to be
> able to use TLS1.3 effectively.

TLS is just a transport for HTTP. There is no reason (nor should there
be) for any difference in semantics of the HTTP messages than over
TLS/1.2 or TCP plain-text transports.

> 
> In our Facebook mobile apps we annotate all the requests that our app
> makes as retry safe or not and that helps us decide how retry
> policies should behave for each request. It's been very useful for
> request reliability.

Why not use the REST semantics of HTTP itself as the signal?

> 
> Would the HTTP-wg be interested in a document describing retry-safety
> and its limitations?

I'm certainly interested to find out why HTTP semantics are suddenly so
insufficient that its needed. IME Facebook was turning into one of the
better HTTP participants before it abandoned for HTTPS-world.

Amos

Received on Tuesday, 22 March 2016 07:33:42 UTC