Re: [webrtc-pc] WHATWG streams for data channel messages

I'm adding some examples here, so it's more clear how my idea would look like from a user's perspective:

### Today's API

Peer A's code:

```js
const dc = pc.createDataChannel(...);
dc.send(largeFile);
```

`largeFile` may be a `Blob` in which case the data doesn't have to be in memory at the time of sending (IIRC, please correct me if I'm wrong) or an `ArrayBuffer` in which case the data will need to be in memory.

Peer B's code:

```js
const dc = pc.createDataChannel(...);
dc.binaryType = 'blob'; // ... or 'arraybuffer', it doesn't really matter here
dc.onmessage = (event) => {
    // Note: Ignoring the string case here.
    // ... write 'event.data' to disk
};
```

When the event has been raised, the large file's data has been transmitted entirely which can take a very long time. Also, the data will be in memory at that point.

### Streamed (assuming the file is not a stream)

Peer A's code:

```js
const dc = pc.createDataChannel(...);
const writableStream = dc.createWritableStream();
const writer = writableStream.getWriter();

while (true) {
    // Note: Once a WritableStreamBYOBWriter exists, we could reuse the same buffer
    // over and over again.
    const length = Math.min(
        myPreferredChunkSize, remainingLength, writer.desiredSize)
    const buffer = new Uint8Array(length);
    // ... copy the file's chunk here into 'buffer'
    //    (or create a view if you already have the chunk in memory)
    await writer.write(buffer);
    // ... continue until all chunks have been written and then break
}
await writer.close();
```

If something goes wrong, you'd call `writer.abort()` which would abort sending the message. (We would have to discuss what `.abort` triggers but it should probably close the channel.)

Peer B's code:

```js
const dc = pc.createDataChannel(...);
dc.binaryType = 'stream';
// We only need one small buffer per channel
const buffer = new Uint8Array(myPreferredChunkSize);
let view = new DataView(buffer.buffer, 0);

// Note: The function is declared async!
dc.onmessage = async (event) => {
    // Note: Ignoring the string case here.
    const readableStream = event.data;
    // You could also use the default reader but then the reader would not
    // copy directly into our buffer.
    const reader = readableStream.getReader({ mode: 'byob' });
    while (true) {
        const chunk = await reader.read(view);
        if (chunk.done) {
            break;
        } else {
            view = chunk.value; // We reclaim the buffer here
            // ... write the view's data to disk
        }
    }
};
```

The reader can call `reader.cancel()` to abort receiving data. (We would have to discuss what `.cancel` triggers but it should probably the channel.)

This of course has the advantage that the file's data doesn't need to be in memory at once for both peers and that backpressure is being taken care of. Furthermore, writing to disk can be started with the first chunk received.

### Streamed (assuming the file is also a stream)

So, the last example looked a lot more complicated than what we have now. But if the file could be read from and written to in form of a stream, it becomes pretty easy.

Peer A's code:

```js
const file = ...;
const dc = pc.createDataChannel(...);
const writableStream = dc.createWritableStream();
await file.readableStream.pipeTo(writableStream);
```

Peer B's code:

```js
const file = ...;
const dc = pc.createDataChannel(...);
dc.binaryType = 'stream';

// Note: The function is declared async!
dc.onmessage = async (event) => {
    // Note: Ignoring the string case here.
    const readableStream = event.data;
    await readableStream.pipeTo(file.writableStream);
};
```

### Combination

Of course, peer A and B can just as well use the existing API on one side. It's entirely up to them.

---

There are several other ways to use WHATWG stream reader/writer instances. I've only shown two of them.

Comments? :)


-- 
GitHub Notification of comment by lgrahl
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/1732#issuecomment-357442429 using your GitHub account

Received on Saturday, 13 January 2018 15:13:20 UTC