Re: [streams] Real time source and skipping (#383)

> An example of a stream that could be "on-demand" is a MediaStream for a screen capture source. The logic in the screen capture could be to only take a capture of the screen when the consumer actually has a need for a new frame, instead of as fast as possible.

:-/ this seems like part of the confusion where MediaStream is used as an opaque handle to anything vaguely media-related, despite actually very different underlying behavior. I don't think it's going to be possible to generically adapt all MediaStreams into a ReadableStream using a single algorithm. For example, my [MediaStreamRecorder](https://domenic.github.io/streaming-mediastreams/) is focused around an algorithm for recording of real-time MediaStreams, and is probably less appropriate for screen capture or video-file backed MediaStreams.

Honestly, what you describe barely sounds like a stream at all. It sounds like a function call: takeScreenshot(). Repurposing ReadableStream's read() method as your takeScreenshot() seems a bit dodgy.

> Being able to know if the read operation would pull from the the internal queue of the stream or pull from the source would allow to read from the stream until the latest queued value, without the risk of making a read call that would block until a new value is actually generated by the source. An "opportunistic read" where we ask that the read operation only returns a queued value would work too.

These both seem like they're exposing way too much of the internal implementation details to the consumer. They shouldn't care about whether they're reading from the queue or from the underlying source. The _entire point_ of the queue is to encapsulate the fact that some underlying sources can't keep data around forever, like a stream can. It's not meant to be exposed.

I'm much more interested in solutions that involve either additional hooks to the underlying source, or parameters to the creation of the stream, to allow the *producer* to decide what it puts into the queue. So again, we'd likely end up with different types of ReadableStream for the different use cases: if the consumer wants no-data-loss recording, they use createMediaStreamRecorder(). If they want sporadic snapshots, they use createMediaStreamSnapshotter(). Both can return ReadableStreams, or they can return appropriate subclasses of ReadableStream.

> While I agree that streams are not appropriate for all use cases, the part that confuses me is that the stream makes assumptions about what the consumer intends to do with the values, i.e. take and concatenate all of them. While this is a probably the most common use case, it's not the only one.

Well, it's kind of definitional. Saying that there are other use cases for streams is actually saying "there are other use cases for this thing that I think of as a 'stream'". It doesn't really say anything about streams as designed here. At some point we do have to draw a line and say what streams are.

That said, I doubt we're really that far off that your use case falls outside streams entirely... I just wanted to illustrate that the changing-single-value actually is a different type of "thing" than a stream:

> For a same source / stream, one consumer might want to process all values, while another consumer might want to process only some values based on some external logic

This is fine and works fine with streams. The second consumer just has to be ready to read from the stream and throw away the values it doesn't care about. Not a problem.

That's a different story than the changing-value paradigm, where logically there isn't even a stream of values, there's a single value and there's updates to that value. Such scenarios do deserve separate APIs, IMO.

> Again, the "recording" or "live" aspect you're referring to in this case is what the consumer decides to do with the stream. 

Again, I disagree. I think different consumers would create different types of streams for these use cases. `new MediaStreamRecorder(mediaStream)` (from the above draft) vs. `new MediaStreamSnapshotter(mediaStream)` (from some other yet-to-be-written draft).

> Without a "caching method", you would have to manually pull like this:

Not necessarily. The producer could take care of this for you, using very similar code. Yeah, it's a bit of a bummer you don't end up using the internal queue. But at least this way the internal queue properly reflects the actual stream of data that the stream consumer sees, which is how it was designed to be used.

> If you want some backgroud on how this issue came up to be, have a look at the thread called "Re: Add "MediaStream with worker" for video processing into the new working items of WebRTC WG" on the W3C media-capture list.

Oh, very cool, I didn't realize this was connected to that proposal! I'm really happy you are looking in to this, and sorry if I come across as uncooperative. I promise, I really do want to make this work, and am just dialoguing to try to work out the apparent impedance mismatch. I appreciate your advocacy for a streams-based solution over there! When I saw the draft, full of workers and such, I was unsure how to integrate streams, but you've given me hope :).

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/383#issuecomment-130472727