on HTTP/QUIC and HTTPBis

On Thu, Mar 9, 2017 at 9:59 PM, Adrien de Croy <adrien@qbik.com> wrote:

>
> for the benefit of those of us who maybe aren't so familiar with QUIC, are
> there any good resources you can refer us to which deal with the bigger
> picture such as why even have QUIC?
>
>
Thanks Adrien.

[minus quicwg, who doesn't need to hear a summary of their own work :) -
and changing the subject line to intentionally break threading]

I'm going to include an *opinion* here on the value of the quic work from
the perspective of the http ecosystem. I think its useful to have something
like this that isn't flavored quite as strongly by the Google experience as
most of the material out there is. (though we can only thank them for their
work and experience - its simply that diversity is valuable).  An important
element of the quic-wg charter is that they work closely with httpbis -
let's consider this part of that cooperation.

tl;dr; There are motivators both large and small - but for me the big 3
are: better perf under high packet loss, lower startup latency, better
streaming. This is largely the transport work leftover from http/2 that
couldn't be done in the TCP/TLS context this wg chose for h2.

A little bit of history: h2 made two really fundamental design choices. One
was to preserve HTTP/1 semantics (i.e. focus on transport issues), and the
other was to constrain that work by what could be done in a traditional
TLS/TCP context. This latter choice wasn't a forgone conclusion - there was
lots of side chatter during the pre-h2 bakeoff phase about whether anyone
would propose something more along the lines of sctp/udp/[d]tls (where RTC
Datachannels was headed contemporaneously). In the end folks felt that
experience and running code were important properties for h2 - building it
on tcp/tls lowered the risk of boiling the ocean, improved the latency of
getting something deployed, and captured meaningful low hanging fruit. For
the issues that it couldn't address there could always be another protocol
and we took seriously the need to create an effective means of versioning
to facilitate that. Rough consensus for a tcp/tls scope was obvious when
all the proposals turned out to be tcp/tls based :)

I think our operational experience has proved that to be a good decision.
h2 is a big win for a lot of cases, it has good uptake, its non-standard
predecessor has receded, and I don't think we could have done too much
better within a TCP/TLS context. Additionally, it can serve as a bootstrap
for 'another protocol' and the means for negotiating that via Alt-Svc and
ALPN which were both defined by the same effort if not the same rfc. That
'other protocol' turns out to be quic and its a good sign for the vibrancy
of http/2 that the quic charter explicitly seeks to map that h2 into its
ecosystem. QUIC seems to be taking on exactly the items that were foreseen
by the scoping of h2.

The chief performance frustration with h2 happens under higher than normal
packet loss. The in-order property of TCP spans different http transactions
in h2 - so a single packet loss in one message prevents subsequent
independent transactions from being delivered until the loss is repaired
(i.e. 2 images in 2 different packets with 1 packet loss can delay them
both.l). At the tail of high packet loss this makes h2 performance regress
from h1. Our community has seen some good data on this recently from Akamai
and Fastly. QUIC fixes this - it multiplexes multiple streams onto one
connection in a way very familiar to those used by 7540, but gives each
stream its own ordering context as if they were independent tcp streams so
they can be delivered independently.

Much of the work of the quic/h2 mapping is disentangling the parts of h2
that utilize the in-order property of the connection. .particularly the
relationship between control data and streams. hpack obviously has some
issues, and the extension-ack as a checkpoint is a problem too. Also
lifecycle counting issues (how do you count the number of open streams if
you can open them without permission but they can reordered around closes?)
etc.. Mike Bishop is doing a lot of the heavy lifting on this (and will
present for you all when we meet in Chicago - so come with opinions and
send pull requests to the quic wg! or at least read the document :)!) We
owe Mike our thanks and support for lots of careful work.

The other major h2 performance issue addressed by QUIC is startup latency.
Essentially it gives you common secure 0RTT connections for previously
visited hosts. You can just fire off N requests to a known host without
blocking on handshaking at the TCP or TLS layer while maintaining a very
small amount of closed state for the host (just a little TLS state, no H2
state at all). Congestion control is the only practical limit. Responses
come back in 1RTT. This is also resilient to things like NAT rebinding. A
world in which you deployed TCP Fast Open with TLS 1.3 Early Data would get
you some of the way to the same goal, but you would still be limited to
just a single packet in that first round trip and the h2 data would have to
share room in that packet with the TCP SYN and the TLS Client Hello - that
1 ethernet sized packet gets crowded real fast.

Speaking of improved latency, quic addresses a common misalignment between
typical streamed http data and TLS. QUIC framing is cognizant of UDP
boundaries and the crypto operations are done on a per UDP basis. The
implications here are important - when doing TLS in a TCP context the
atomic crypto operation is done on a TLS record (of relatively arbitrary
size - but 4k, 8k, 16k are common). Due to the packet agnostic byte-stream
interface of TCP, unless you worked amazingly super duper hard at it these
records commonly span packet boundaries - but the delivery time of the
application level data (e.g. progressive images) was dependent on the
arrival of the last packet in the record. The application would incur ~a
round trip of latency rendering partially received data whenever a
congestion window limit was hit even without loss or reordering.. in the
presence of loss or reordering it can get really bad. Ilya has done some
great writing on this topic in the past - but implementations still
commonly do it poorly. QUIC automatically optimally aligns the crypto with
the UDP boundary (and has some PMTU magic that will generally ensure UDP is
not fragmented).

So those are the big 3: more stable under high packet loss, lower startup
latency, better streaming. They're really big :) But there's more -

Blending the application and the transport has some advantages - I think
the chief one here is that the application is going to have a much better
idea of how to manage its buffers as it will have closer access to ack
feedback loops. We've seen this several times in H2 implementations where
code doesn't know how to gauge a BDP worth of data and ends up creating
huge amounts of buffering which impact its own ability to multiplex,
cancel, and honor priority (which are afterall the major features of h2).
The result can be horrible responsiveness. We will have a presenter in
Chicago talk about some related academic measurements around this topic as
part of our effort to share feedback on the working group's past outputs.
This is a subtle thing as I think you can do approximately-the-right-thing
in both h2 and quic, but the quic architecture should make it easier to get
quality implementations.

I also expect that a lot of the weird corner cases that result in data loss
in h1/h2 around TCP RST will be eliminated. (RSTs are often routinely
generated by application errors and sometimes result in silent kernel
discard of data that has already been received and acked but not yet
delivered to the application - this combined with large TLS records can
sometimes result in the loss of a lot of data. Its a pretty weak part of
TCP.)

Further, TCP Tuning for HTTP (1 or 2) is operationally complicated. One of
the complications is the administrative separation between transport and
application. How do you set IW? Are you doing TLP? RACK? Is Fast Open an
option for you? QUIC, because it can be bundled with an application in
userspace, provides a deployment option for getting best of breed transport
algorithms deployed to hosts that may be running legacy TCP stacks. You can
expect a more consistent "best practices" experience across platforms and a
pivot for iterating on new designs and ideas for evolution. Speaking of
evolution, quic is almost entirely encrypted and basically fully
authenticated. This intentionally exposes very few bits to the network with
the intention of preventing ossification around patterns introduced in v1
the way things. Ossification has been a huge problem for TCP and http and
we'd like to do better.

I'll wrap it up by noting that blending transport and crypto has some
advantages beyond packet alignment too - perhaps my favorite is that it
seems likely quic will be resistant to RST injections and the modification
of basic congestion control by things like peps. (you may or may not
consider than an advantage - I'm aware.) Though it will be vulnerable to
downgrade attacks to TCP in the same way h2 is vulnerable to downgrade
attacks to h1 :(

I think the question for our working group is how we want to engage with
all this change - as part of that you can expect regular updates about the
quic/http mapping to be part of our process for a while and Mike is doing a
great job of making clear choices like how quic interacts with the legacy
registries more visible in this audience. I'd like to make it possible for
those with only an interest in the HTTP portions of this (to whatever
extent that is possible) to be able to share their experience without
joining the full QUIC-WG firehose - let Mark and I know if you have any
thoughts on how to further facilitate that beyond what we're doing.

So far our experience with QUIC (as a proxy) has been to simply block it,
> since it provides a bypass mechanism for proxy control.
>
>
Obviously this hasn't been a proxy centric review, but there is no
fundamental reason a client can't speak QUIC to a proxy in the same way
some of them support h2 to the proxy now. These are not changes in
semantics. I would hope to see it for any client with forward proxy support
but it might not be the first thing rolled out.

-Patrick