Re: [streams] Real time source and skipping (#383)

@mhofman thank you for opening this very interesting thread, and my apologies for not engaging in it earlier (last week was TC39). I agree that in general this is a tricky issue as streams are generally designed for sequences in which every chunk is meaningful and none should be missed. But I am confident we can figure it out with some work.

> on demand, they generate a new frame only when needed

Can you give an example of such a scenario? Is the idea e.g. reading a movie file from a disk?

> The source controller could do some skipping by not queuing a new frame if the queue of the stream is full, but this doesn't provide the desired behavior since we would be skipping new frames, keeping old ones in the queue

A size-one queue gives very close to correct behavior. But, it would keep a single old frame before skipping to new ones.

> Similarly, the consumer cannot drain the queue of the stream when it's done processing a frame

Related: #324, the idea that a consumer should signal to the readable stream when it's actually processed a chunk.

> Should be consider adding a method on readable streams to probe the size of the queue or behavior of the read?

I don't really understand how this would solve the issue.

> Should a caching strategy be added to readable streams that would automatically discard elements from the queue under some conditions?

This seems like a more interesting approach. If it weren't for the exclusive reader semantics, I think it could be accomplished already: the stream would simply read from itself if `controller.desiredSize <= 0`, in order to drop the oldest chunk. I think that is what @wanderview is alluding to with a transform stream or similar wrapper.

> I thought about a Transform but I'm having a hard time seeing how that would work:

It seems like you'd need different transforms for each type of source. This comes back to my question of what use cases an on-demand stream is. It doesn't seem unreasonable to distinguish between webcams and movie files. I agree it would be nice if you didn't have to, though.

> PS: I'm using the example of video frames as it's relevant to me in this case, and is easy to comprehend, but really this applies to any problem where we want to skip intermediate values in the stream and pass through the "last-in" when pulling. ... Another use case of a real-time push source and skipping intermediate values but the latest, would be a "stream of cursor positions".

This kind of talk in general makes me unclear whether streams are really a correctly matched solution to your problem. The primitive here is something more like "changing value," which has a single "latest value". Maybe it also optionally has a stream (or something) of notifications that the value has changed. The consumer can then go check the latest value, either based on the notifications (if they are present) or based on polling (which seems like a more likely architecture). See also [are readable streams appropriate for my use case?](https://github.com/whatwg/streams/blob/master/FAQ.md#are-readable-streams-appropriate-for-my-use-case)

> Now as a consumer, if I don't read while new values are produced, I want to be able to "fast-forward" to the latest value in the stream, without having to wait for a new value to be produced by the source. Since there is no way to know if a read from a stream will just pull from the queue or wait for a new value, this is not possible to accomplish.

Again, this seems based on flawed architectural assumptions. To render video, you have a frame-budget (24 fps, or 60 fps, or whatever). Whatever the latest frame is that's come in during that time, you want to use that. So I'd expect something like

```js
let currentFrame;

// shorthand for the usual async loop, except we cap it
// to not pull more than 60 times per second.
pumpStream(rs, chunk => currentFrame = chunk);

// elsewhere in the code
requestAnimationFrameLoop(() => render(currentFrame));
```

You seem to have a very different architectural model in mind, something like

```js
let currentFrame;

requestAnimationFrameLoop(() => {
  if (!rs.wouldPull) {
    currentFrame = rs.read();
  }
  
  render(currentFrame);
});

rs.rememberToDropFramesIfIDontAskForThemInTime();
```

But this seems to unnecessarily couple your rendering loop with your consumption loop.

---

I'll avoid getting in to the question of specific API proposals to address your use case until I better understand it. But there are several that sound workable on the table right now. I am curious how you respond to my pseudo-code though.

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/383#issuecomment-127380859