Re: draft-kinnear-httpbis-http2-transport questions

Hi Martin, 

Thanks for the read through and for the comments! 

You’ve managed to hit upon most of the open questions remaining for this approach, which I expect will be discussed in Prague in tsvwg and httpbis. 
My intent here is to have this document present (alongside its -00) to help discuss those questions, at which point we can rev the draft and start a more detailed discussion on the list with a more concrete proposal. 

> On Mar 20, 2019, at 3:05 AM, Martin Thomson <mt@lowentropy.net <mailto:mt@lowentropy.net>> wrote:
> 
> Lots to think about here.  Thanks for sharing.
> 
> I have a few fairly basic questions about the goals of the draft.
> 
> Why do you need to use both :protocol and a setting?  Isn't SETTINGS_ENABLE_CONNECT_PROTOCOL plus the new values for :protocol enough?

Short answer: Yes, just doing a new :protocol value with the existing setting would be fine. There’s a big caveat around the “bidirectional” part, and I left the setting as covering both, when it probably should be only the second part. 

Long answer: 
So this brings me to some of the “where do we want this to go” questions. Right now, there are a few potentially competing goals, and I’ve left the setting to cover everything to ensure that any endpoint supporting this supports all of it. I expect that’s not what we really want long term, and that this can be split into two parts.  

A brief summary of those goals (this will be covered in a presentation in tsvwg this week): 

- Ability to establish transport of arbitrary bytes over a single stream on a multiplexed H/2 connection. At the API layer, we present this as if it were a TCP/TLS stream, with all the associated state machinery plumbed, but underneath we’re sharing a single H/2 connection. 

- Ability to share that H/2 connection with other streams, potentially also transporting arbitrary bytes, or maybe they’re performing standard H/2 requests/responses by sending HTTP messages. 

- Ability for the server to establish streams that transport arbitrary bytes (note that the client can reject these incoming requests for streams, just like any other H/2 stream). 
 This one is where it starts to get tricky, since we want everything to be bidirectional. This is the main reason to have a SETTING, since it’s a much larger leap than just adding a new :protocol value, and I’m not convinced that it’s currently allowed/even reasonable with CONNECT or any HTTP message. This is covered in -00 rather well by STREAM frames which are just defined as a new, bidirectional concept, but I’m not clear that -01 adequately solves this. (And ideas on the best way to accomplish this are most welcome!) 


In terms of use cases, we’re really looking at the intersection of a few things, and it may eventually be better to split this into two documents. 

1. Arbitrary transport of bytes (and potentially a framing layer that suffices for datagrams, see comments below) is useful for tunneling things over an H/2 connection while getting all the benefits of the shared underlying transport. 
 This could enable nice types of proxies, like for UDP, using existing CONNECT (not to the other endpoint), etc. 
 This is great to enable talking directly to the other endpoint if you wanted to bundle everything to that host together (of course, HoL blocking is still a thing)
 There are a several more that are written up across some of the HiNT (https://tools.ietf.org/html/draft-pardue-httpbis-http-network-tunnelling-01 <https://tools.ietf.org/html/draft-pardue-httpbis-http-network-tunnelling-01>) and HELIUM (https://tools.ietf.org/html/draft-schwartz-httpbis-helium-00 <https://tools.ietf.org/html/draft-schwartz-httpbis-helium-00>) drafts, which have lots of really great ideas and discussion.
 Having headers available in the HTTP message (as opposed to the -00 with STREAM frames) is useful for these types of use cases. 

 This starts to cover some of the “enhance CONNECT a little more and define ways to send the missing pieces over it/negotiate how to have both ends agree on what’s happening” strategy. 
 

2. Bidirectional establishment of arbitrary transport of bytes is another very useful concept
 I’d like to have both ends of a connection be able to establish a new stream that carries bytes — once you’re connected to someone, both sides should be able to initiate contact with the other. I use this today for making just a single link between devices and then having different services run on top of that shared link. 
 This is something that’s offered by QUIC Transport (i.e. not H/3) and there are lots of use cases for this - however we likely need to fall back to TLS/TCP for some percentage of connections. HTTP/3 falls back to HTTP/2 over TLS/TCP and is happy. QUIC Transport needs something to fall back to, and HTTP/2 Transport could be that thing.

For QUIC Transport falling back to TLS/TCP, some frame types aren’t necessary (ACK) since TLS/TCP provides reliability, etc. 
At that point, pretty much every frame maps to H/2, with the exception of being able to open new streams (and from either endpoint) without HTTP headers. I’m thinking that something along the lines of the -00 for this document works a bit more cleanly for this purpose, but that’s the main point of the discussion to be had. A QUIC STREAM frame would map pretty much directly to the STREAM frame in -00 and we can decide to use DATA for application data or keep it in the STREAM frame like QUIC. 


In terms of future direction, I’m thinking that this may need to be two documents, one for “how to run QUIC transport over TLS/TCP using H/2 framing / H/2” and one for “adding new :protocols to the extended CONNECT handshake”. The current document clubs beginnings of both together rather harshly and I’m not sure that’s helpful or useful. 


> 
> How does a :protocol of bytestream differ from a plan CONNECT?

My take on this is that RFC8441 specifies a way to use CONNECT to establish a stream terminated by the other endpoint of the H/2 connection, whereas plain CONNECT establishes a new connection to (potentially) a different host entirely and then tunnels traffic through to that other host.
Because of that, you want many of the same things that WebSocket over H/2 needed, like ways to match up listening applications to these incoming streams.

> What is the point of a :protocol of datagram?  Is the idea to use UDP?  The string "UDP" appears nowhere in the draft, so I've concluded that that isn't the case, except that using UDP would be the only way in which this would be different (and therefore marginally useful, modulo the obvious head-of-line stuff).

I debated dropping “datagram” out as a potential :protocol value, but left it in for now, since one of the things that would be nice to have is the ability to encapsulate datagrams. 
The idea here was to support sending delineated messages, but if you really want to do that you should probably be adding some framing (or even WebSockets) if you’re coalescing DATA frames, etc. At that point, we really need some framing in here (which isn’t in -01) or to remove it entirely.

Just to make sure, by “use UDP” do you mean this would be H/2 over TLS encapsulating datagrams on a given stream, or using UDP/DTLS underneath and sending the H/2 frames over that?

> 
> FWIW, the datagram thing might work, but DATA frames aren't really designed to work that way.  In order to use a datagram tunnel effectively you would have to change the complete stack for DATA to avoid coalescing of frames, buffering, splitting and all those sorts of things that are natural for something with an underlying assumption of streaming semantics.  It might be better to define a new frame type for this purpose.

This is a really good point, since DATA frames aren’t guaranteed to get to the other side in the same shape that you tried to send them. If we’re going that far, then yes, additional frames may be easier to specify rather than changing the rules based upon the presence of an extension. 
Alternately, we go with yet-another-framing-layer and stick with that approach. 

> If the point is to enable UDP, then there are a few features I might want to request.  For instance, the ability to setup a listener.  The ability to know what the IP and port of that listener are.  The ability to set rules about what remote addresses can send to that listener (other than a fixed, CONNECTed peer).  That might get closer to being useful for something like WebRTC.  I would want all of that for QUIC, for instance, where head-of-line is less of a concern.  But that gets dangerously close to TURN, so we'd have to have the NIH discussion in light of that.

Would you communicate to the other endpoint about the listener or is that more of an API consideration? (All valid here, just curious)

Thanks again for the thoughtful review, and I’d love any ideas on how to get this where it needs to go! 

Thanks,
Eric


> 
> Cheers,
> Martin