Re: [streams] Operation stream (#287)

> merge of underlying sink/source, writable/readable stream into OperationStream

This is an interesting one. I am still pretty sure this will not be good for the user-facing API. That is, I think the current user-facing API, both for stream creators and stream users, is pretty optimal. In particular the dual start() and pull() hooks for readable streams, and the write() and close() hooks for writable streams, are quite nice for stream creators.

I think we should consider the API separately from the implementation, however. If it turns out that a two-sided stream implementation is conceptually simple and makes the spec and implementation easier to do, then that is good, as long as the API niceness is preserved. Conversely, if there are API improvements to be made, we can discuss them separately from discussing the strategy used to implement them.

Another thought: it might be possible that the work here is going to be most valuable when considering how to spec and implement TransformStream. Currently TransformStream is kind of a hacked-together thing where I reuse most of the ReadableStream and WritableStream logic. However it is probably more efficient to implement it as a single primitive, which happens to expose the ReadableStream and WritableStream APIs. OperationStream seems kind of like that approach. (Or, rather, a special-case of it, where the transform is the identity.)

> precise flow control stuff

This is the kind of API improvements stuff that I think we need to consider on its own merits, outside of the "implementation detail" of two-sided pipe streams. More below :)

> ...
> Based on the observations above, I tried to merge underlying source/sink
> ...

These are really interesting observations. Can you explain more why a readable stream would need acknowledgement/error for each chunk read? Currently we kind of have acknowledgement---when you call read(), you implicitly acknowledge the data. And when you cancel the stream, you indicate some kind of error. (I am not sure why you would need to indicate error on an individual chunk.)

> I also designed and implemented some precise flow control stuff using this opportunity. Sorry for confusing but they're independent parts.

This is the stuff I really want to learn about!!

space/waitForSpaceChange are on writable stream. I guess they are more sophisticated versions of the current backpressure signal, of "writable" or "waiting", that can be used to communicate _how much_ can be written. This makes sense.

I think it mostly makes sense for WritableByteStream, because then when piping you could ask for as many bytes from the readable byte stream as `space` indicates on the target writable stream. Maybe it could make sense for general readable streams using their `size()` strategy method, but that is kind of complicated. E.g. it assumes compatible notions of size between the source stream and the target. Hmm.

I don't quite understand window though. I imagine it maps to TCP window stuff but I don't understand that perfectly either :(. Could you help me understand it? And, how general is such an idea---just byte streams, or anything with a size, or...? Does it depend on the capabilities of the underlying source, so e.g. it works for sockets but not for files?



---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/pull/287#issuecomment-76333828