W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > January 2018

Re: [webrtc-pc] WHATWG streams for data channel messages

From: Lennart Grahl via GitHub <sysbot+gh@w3.org>
Date: Wed, 17 Jan 2018 22:36:41 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-358472902-1516228600-sysbot+gh@w3.org>
I will try to explain why the second point is important (even for WebSocket) because I believe you're not alone with that perception. Consider the following scenario: I want to receive two files.

Let's take [this example](https://streams.spec.whatwg.org/#example-rs-push-no-backpressure) as a starting point...

```js
const file = <some stream>;
const readableStream = makeReadableWebSocketStream('wss://example.com:443/', 'protocol');
readableStream.pipeTo(file.writableStream);
```

Let's assume the sender will send each file as a separate stream. So, this is going to store the first file but after that the WS connection is closed. What can we do? Well, we could persuade the sender to send both files inside the same stream...

```js
const file = <some stream>;
const readableStream = makeReadableWebSocketStream('wss://example.com:443/', 'protocol');
readableStream.pipeTo(file.writableStream);
```

Meh, now we've stored both files as one file. Unfortunate. What can we do? Well, we could add a header and include the file's size... or we add a header (or footer) to each chunk that tells us whether the file is EOF or not. Usually, we will prefer the latter because the first idea requires us to know the file's size. Something we may not always know (at the latest when we want to pipe another stream that doesn't follow our framing protocol). So, let's go for the latter...

```js
const readableStream = makeReadableWebSocketStream('wss://example.com:443/', 'protocol');
const reader = readableStream.getReader();
let file = <some stream>;
let writer = file.writableStream.getWriter();

while (true) {
    const chunk = await reader.read()
    if (chunk.done) {
        break;
    }
    const header = new DataView(chunk.value.buffer);
    const eof = header.getUint8(0) !== 0;
    await writer.write(new Uint8Array(chunk.value.buffer, 1));
    if (eof) {
        await writer.close();
        file = <some stream>;
        writer = file.writableStream.getWriter();
    }
}
await writer.close();
```

To recap what we've done right now: We have added a proprietary framing protocol on the application layer which essentially **encapsulated two different streams** (which is why we couldn't use `.pipeTo` anymore). This added complexity and required us to inspect the chunks because it is yet another protocol. It also required the sender to copy the header (or footer) into the buffer. It will have quite an impact on throughput (ping me if you want to know more, I've written some tests a while ago) and it was also unnecessary because the underlying transport already provided such a framing protocol we could have used for this exact purpose. It just wasn't doable with the existing API.

(I have completely ignored that this of course wouldn't work for data channels that aren't reliable and ordered. For this, you will need to add a more complicated framing protocol such as [this](https://github.com/saltyrtc/saltyrtc-meta/blob/master/Chunking.md).)

With the approach suggested in this issue carried over to the WebSocket API, it would be as simple as this...

```js
const ws = new WebSocket('wss://example.com:443/', 'protocol');
ws.binaryType = 'stream';
ws.onmessage = async (event) => {
    // Note: Ignoring the string case here once again.
    const file = <some stream>;
    const readableStream = event.data;
    await readableStream.pipeTo(file.writableStream);
};
```

Was that helpful? :smiley: 

-- 
GitHub Notification of comment by lgrahl
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/1732#issuecomment-358472902 using your GitHub account
Received on Wednesday, 17 January 2018 22:36:45 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:21:58 UTC