Re: [streams] Readable byte streams should support an internal queue (#353)

The boundary between the underlying byte source and the readable byte stream doesn't need to correspond to any specific layer boundary. It just depends on the philosophy the author of the spec gave to the spec. When we want to drain some data to one world (storage, kernel, process) from another (another storage, userland, another process), they're always allowed to do that in various means.

Having some strategy-controlled buffer example in the spec is good. But the spec shouldn't impress the readers that they must implement the buffer in JavaScript. My suggestion is having two separate classes. One is just describing all the publicly visible requirements (including invariants, interaction between methods, properties, etc.), and the other is a strategy-controlled buffer which is convenient for push source implementor. The buffer class is good example for explaining how to interpret backpressure and controlling buffer size, data generation in general. That can be referred to when writing non-JavaScript buffering components. And of course, it's also useful as a built-in JS library.

I want the separation especially for byte stream. If the source is push source, then it's convenient for the implementor of the source if the stream has a built-in buffer. In that case, the source doesn't understand BYOB style pulling, so we need to take care of the gap by copying the contents to the view provided by the consumer. But this is not expected for a source that understands BYOB. For BYOB read requests, the source just wants to get the view and fulfill the corresponding pending request with the view (possibly detached) filled with generated contents as-is. There shouldn't any automatic copy. Having a queue even for BYOB-capable source may be useful. But the queue system for BYOB-capable source and one for push source would be very different.

I try to finish prototyping the ideas explained above. WIP branch is at https://github.com/whatwg/streams/blob/bytestream/index.bs.

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/353#issuecomment-99894225