[streams] Real time source and skipping (#383)

Let's suppose we have a source that generates frames. These sources can be of 2 types:
- real-time, they generate frames constantly no matter what, e.g. a webcam. These sources would probably use push semantics
- on demand, they generate a new frame only when needed. These source could be implemented either with pull semantics, or push with pause/resume back-pressure.

Streams are a great way to unify the consumption of these sources. It works great when the consumer wants to process every frame produced by the source. However things don't go as well in the case where the consumer is not able to process all the frames of a real-time source.

What a consumer generally wants in that case is to be able to skip frames produced while it's currently  processing one.
Currently the back-pressure mechanism of readable streams doesn't help since it's informative only. Frames will keep getting queued into the readable stream for pure push sources, no matter the size of the queue. The source controller could do some skipping by not queuing a new frame if the queue of the stream is full, but this doesn't provide the desired behavior since we would be skipping new frames, keeping old ones in the queue. It's preferable to discard old frames and keep new ones only.
Similarly, the consumer cannot drain the queue of the stream when it's done processing a frame, and keep only the last one generated, as there is no way know the status of the queue as a consumer or know if the read operation will pull from the queue or wait for a new chunk to be available.

Can anyone suggest how to solve this use case?
Should be consider adding a method on readable streams to probe the size of the queue or behavior of the read?
Should a caching strategy be added to readable streams that would automatically discard elements from the queue under some conditions?

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/383

Received on Thursday, 30 July 2015 05:19:48 UTC