Re: [whatwg/streams] Is when pull() is called on an underlying source deterministic? (#1155)

Each individual stream fills up to its own `highWaterMark`, pulling in chunks as necessary. So if you have a pipe chain like this:
```javascript
const rs1 = new ReadableStream({
  pull(controller) {
    console.log("pull");
    controller.enqueue("chunk");
  }
}, { highWaterMark: 5 });
const rs2 = rs1
  .pipeThrough(new TransformStream({}, { highWaterMark: 2 }))
  .pipeThrough(new TransformStream({}, { highWaterMark: 1 }));
```
then this will log `"pull"` 8 times (5 + 2 + 1) to fill up all queues. If you then start reading from `rs2`:
```javascript
const reader = rs2.getReader();
await reader.read();
```
then you'll get a chunk out of the last stream's queue. Since that causes its queue to dip below its high water mark (1), it'll pull a chunk from the previous stream. The same thing happens with this stream: a chunk is pulled from its queue, it dips below its high water mark (2), and pulls a chunk from `rs1`. `rs1` immediately provides a chunk from its queue, also dips below its high water mark, and pulls from its underlying source.

In general, I don't think it's possible for a `ReadableStream` to know when "the final consumer" has received a previously enqueued chunk. That chunk can still be in the queue of a transform stream further down the pipe chain (as shown above), or inside the internal state of a `transform()` method that is currently processing the chunk. I think the only way to do this is to actually make the final consumer (a `WritableStream` acting as a sink, or a destructive `TransformStream`) manually `close()` the chunk when it (eventually) receives it.

On a slightly related note: what would you expect to happen when you enqueue a couple of `VideoFrame`s and then the stream becomes canceled? The Streams standard will discard all queued chunks, but then it's up to the garbage collector to clean up any resources associated with these `VideoFrame`s. Can that cause problems if these are not cleaned up soon enough? 🤔 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/1155#issuecomment-891878727

Received on Tuesday, 3 August 2021 14:09:30 UTC