Post before acking settings?

In OkHttp we’re investigating a bug
<https://github.com/square/okhttp/issues/2506> where nginx v1.9.15 sends us
REFUSED_STREAM when we send our HEADERS and DATA frames before acking the
server’s SETTINGS.

Should a well behaved HTTP/2 client wait to ack the server’s SETTINGS
before sending other frames? The spec says not waiting is okay
<http://http2.github.io/http2-spec/index.html#rfc.section.3.5>:

To avoid unnecessary latency, clients are permitted to send additional
frames to the server immediately after sending the client connection
preface, without waiting to receive the server connection preface. It is
important to note, however, that the server connection preface SETTINGS
frame might include parameters that necessarily alter how a client is
expected to communicate with the server. Upon receiving the SETTINGS frame,
the client is expected to honor any parameters established.

I’d love advice on how to resolve this. My options as I see them:

   1. Change the client to ack SETTINGS before sending other frames.
   2. Petition HTTP/2 server implementors to not refuse streams that
   precede the SETTINGS ack.
   3. Recover when we get a REFUSED_STREAM.

Note that recovering from a REFUSED_STREAM isn’t an easy win for us. OkHttp
doesn’t retain the (potentially large, potentially sensitive) request body
bytes that were posted. To recover the application layer will be prompted
for new request body bytes. This is typically just serializing some JSON.
But it could be invasive, like prompting a user for their fingerprint so
that a device’s secure element can be used.

Thanks!
– Jesse
​