W3C home > Mailing lists > Public > public-ortc@w3.org > October 2017

QUIC: recent developments

From: Bernard Aboba <Bernard.Aboba@microsoft.com>
Date: Mon, 2 Oct 2017 18:12:52 +0000
To: "public-ortc@w3.org" <public-ortc@w3.org>
Message-ID: <CO2PR00MB01364A65052C5F04D2264EA1EC7D0@CO2PR00MB0136.namprd00.prod.outlook.com>
Specification work on the QUIC protocol is continuing within the IETF QUIC WG, and a number of implementations are now in progress. 

With the QUIC WG hosting an interim meeting in Seattle this week, I thought I'd summarize where we are with QUIC support in the ORTC API, and mention a few recent developments that may be relevant to that work. 

At this point, we have incorporated an RTCQuicTransport interface into the ORTC API:

However, while the current "Big Picture" mentions QuicStreams, we do not yet have a section defining the QuicStream API. 

In addition to Peter's proposal for a QuicStream API, EKR has recently posted a (C) QUIC API proposal on the QUIC WG mailing list, which bears some similarity to Peter's proposal: 

So far, work in the IETF QUIC WG has focused on defining QUIC support for reliable transport: 

This has lead to some questions about how QUIC can be used for unreliable transport (e.g. hosting an unreliable RTCDataChannel interface over QUIC).   Recently a draft has been submitted on the subject of unreliable transport: 

Aside from these notes of progress, there is at least one major sticking point.  Multiplexing of QUIC with other WebRTC protocols (e.g. RTP, RTCP, STUN, TURN, DTLS, ZRTP, etc.) has not yet been defined.   Here is a link to the issue raised in the QUIC WG.  An initial proposal (involving changes to the header bits) was rejected: 

-----Original Message-----
From: Lennart Grahl [mailto:lennart.grahl@gmail.com] 
Sent: Thursday, July 13, 2017 6:02 AM
To: Peter Thatcher <pthatcher@google.com>
Cc: public-ortc@w3.org
Subject: Re: A proposal for putting QUIC streams into ORTC

I really like the concept as I believe that we need a streaming API for arbitrary data transfer in the WebRTC environment. And I agree that an optional low-level approach is better than confronting application developers with low-level buffering techniques when they just don't need it (this is where the existing data channel API makes sense).

However, the concept itself is applicable to the SCTP transport as well.
So, personally, I believe it would make sense if we would have a unified DataStream (renamed from your QuicStream proposal) that can work on top of either the SctpTransport or the QuicTransport, so we do not have separate APIs for SCTP and QUIC. The DataChannel would then work on top of the DataStream.

Yes, for such an API, we will have to look at technical differences between SCTP and QUIC. If these are too large for a direct DataStream interface, the QuicStream and the SctpStream interfaces could be just another layer in between. But I believe a well-tailored API should be possible as there are APIs such as the NEAT project's API
(https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fneat.readthedocs.io%2Fen%2Flatest%2Ftutorial.html&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C12990010786e425a9e0108d4c9efe71f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636355479601344448&sdata=SXf5IbQdfrtnB547ymZCnHrCjX5XRc77BPkJAzHVzH8%3D&reserved=0) which unify transport protocols with even larger differences (granted, they don't need the concept of streams but both QUIC and SCTP seem to have it).

Maybe we should even consider creating WHATWG streams in an extended form (that support the concept of unreliable and unordered data
transfer) directly, so the concept and logic of data streams, buffering, back-pressure, etc., can be applied to other transport protocols in the future and they don't need to reinvent the wheel.

I'd be happy to participate in creating such an API.


On 13.07.2017 13:42, Peter Thatcher wrote:
> There has been a lot of interest in adding QUIC-based data channels to 
> ORTC, and it seems like a natural fit to layer some new QuicTransport 
> on top of an IceTransport to produce data channels the same way we do 
> form an SctpTransport layered on top of a DtlsTransport on top of an IceTransport.
> But as I have spent time thinking about it and designing it, I 
> realized that there is a more fundamental primitive that we can 
> expose, slightly lower-level than data channels.  And if we give that 
> primitive an API, it would not only be possible to implement data 
> channels on top, it would be possible to build many other things on 
> top in a way that would be more simple, powerful, and flexible at the same time.
> That powerful fundamental primitive is the QUIC stream.
> Here's as simplified version of the API I propose (I slightly more 
> full one a little later), centered around QUIC streams:
> [Constructor(IceTransport)]
> partial interface QuicTransport {
>   QuicStream createStream();
>   attribute EventHandler onstream; // QuicStream from remote side };
> partial interface QuicStream {
>   // If there is space in the local buffer, write to the buffer.
>   // Otherwise, fail.
>   void write(Uint8Array data);
>   // Will maybe write one last value before finishing
>   // and then a "fin" bit.
>   void finish(optional Uint8Array data);
>   // The remote side has acked everything, including the "fin" bit
>   // Which means it is safe to close the stream.
>   readonly attribute EventHandler onacked;
>   // Reads all that is in the buffer.
>   Uint8Array read();
>   // Fills the array exactly from the read buffer.
>   // If that amount is not available, it fails.
>   void readInto(Uint8Array data);
>   // Closing on either side trigger on close on both sides.
>   // Once closed, all future reads and writes will fail.
>   // Nothing will be sent or received any longer.
>   // Everything buffered will be thrown away.
>   // Closing an already closed stream does nothing.
>   void close();
>   attribute EventHandler onclosed;
> }
> Basically, you can write, read, and close.   You can also "finish", know
> when the remote side has "finished" and know when the remote side has 
> acked everything.
>>From here, as I'll show with some examples, you can send and receive 
> message out of order or with time-bounded reliability.  You can also 
> send and receive large messages with back pressure.  You can build 
> data channels.  Or you can even build WHATWG streams.
> But there's just one last piece to the puzzle to explain: buffering.  
> In order for back pressure to work properly and easily we need a way 
> of causing the app to wait until there is room in a limited-size buffer to
> send and receive.    In the examples, you'll see how this is used, and then
> I'll show the methods to accomplish this.
> Example of sending small unreliable/unordered messages with a short timeout:
> let ice = ...;
> let quic = new QuicTransport(ice);
> let maxMessageSize = 4096;
> let timeout = 5000;
> // Write outgoing message
> let data = ...;
> let qstream = quic.createStream();
> await qstream.waitForWritable(data.byteLength);
> qstream.finish(data);
> setTimeout(() => qstream.close(), timeout);
> // Read incoming messages.
> quic.onstream = qstream => {
>   await qstream.waitForReadableOrFinished(maxMessageSize);
>   let data = qstream.read();
>   // Do something with data.
> };
> Example of sending a large message in chunks, with back pressure let 
> ice = ...; let quic = new QuicTransport(ice);
> // Write out in chunks
> let qstream = quic.createStream();
> let chunks = ...;
> for chunk in chunks {
>   await qstream.waitForWritable(chunk.byteLength);
>   qstream.write(chunk);
> }
> // Read in the chunks
> quic.onstream = qstream => {
>   await qstream.waitForReadableOrFinished();
>   while (!qstream.finished) {
>     let data = qstream.read();
>     // .... Use the data somewhere
>     await qstream.waitForReadableOrFinished();
>   }
> }
> Building reliable, ordered DataChannels on top of streams:
> A reliable, ordered DataChannel can be implemented on top of a single 
> QuicStream with a simple framing mechanism to insert messages into the 
> stream.  For example, a framing mechanism like the following could be used:
> DataChannel.send writes the following to the stream:
> - 2 bits to indicate the type (0x01 == string; 0x10 == binary)
> - 2 bits for the number of additional bytes (N) for the length of the 
> message payload (0x00 = 0, 0x01 = 1, 0x10 = 2, 0x11 = 4)
> - 4 + (8 * N) bits for the length of the message (M)
> - M bytes for the message
> Building unreliable or unordered DataChannels:
> To  send unordered or unreliable messages over QUIC, multiple QUIC 
> streams must be used, perhaps one for each message.  A DataChannel 
> could be built which creates a new QUIC stream for each message which 
> writes a data channel ID as metadata first and then a message write 
> after.  However, an application will typically simply skip this level 
> of abstraction and just use QuicStreams directly for sending and 
> receiving unreliable or unordered messages, since it is more simple that way.
> WHATWG streams:
> WHATWG streams wrap an "underlying source" to create ReadableStreams 
> and WritableStreams which provide some convenient functionality.  We 
> can create "underlying sources" from QuicStreams with something like this:
> var qstream = quic.createStream();
> new ReadableStream(
>   {
>     type: "bytes",
>     start: function(controller) {
>       qstream.onclose = () => controller.close();
>     },
>     pull: async function(controller) {
>       if (controller.byobRequest) {
>         var data = controller.byobRequest.view;
>         await qstream.waitForReadableOrFinished(data.byteLength);
>         if (qstream.finished) {
>           let finishedData = qstream.read()
>           copyArray(finishedData, data);
>           controller.byobRequest.respond(finishedData.byteLength);
>         } else {
>           qstream.readInto(data);
>           controller.byobRequest.respond(data.byteLength);
>         }
>       } else {
>         await qstream.waitForReadableOrFinished();
>         controller.enqueue(qstream.read());
>       }
>     },
>     cancel: function() {
>       qstream.close();
>     }
>   },
>   new ByteLengthQueuingStrategy({ highWaterMark: 4096 }) );
> new WritableStream(
>   {
>     type: "bytes",
>     start: function (controller) {
>       qstream.onclose = () => controller.close();
>     },
>     write: async function(chunk, controller) {
>       await qstream.waitForWritableAmount(chunk.byteLength);
>       qstream.write(chunk);
>     }
>     close: async function() {
>       qstream.finish();
>       await waitForEvent(qstream.onacked);
>       qstream.close();
>     }
>     abort: function(reason) {
>       qstream.close();
>     }
>   },
>   new ByteLengthQueuingStrategy({ highWaterMark: 4096 }) );
> Finally, here are the two buffering methods I mentioned that are need 
> to make the back pressure and buffer work correctly:
> partial interface QuicStream {
>   // Resolves when the amount can be written or buffered,
>   // Up to the maxBufferedAmount (== amount if not given)
>   Promise waitForWritable(unsigned long amount,
>                                           optional unsigned long 
> maxBufferedAmount);
>   // Resolves when the amount can be written or buffered,
>   // Up to the maxBufferedAmount (== amount if not given)
>   // Will also resolved if the stream finished.
>   Promise waitForReadableOrFinished(
>     unsigned long amount, optional unsigned long maxBufferedAmount); }
> I think this approach of QUIC streams with data channels and other 
> abstractions built on top will be flexible, powerful, and about as 
> simple as it can get.

Received on Monday, 2 October 2017 18:13:28 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:40:01 UTC