Re: [streams] Change the model for ReadableStream to have async read() (#288)

@yutakahirano 

> We are about to introduce a pool mechanism (or pending read ops, if you prefer async read) into ReadableByteStream. We should replicate the mechanism into ExclusiveReader and it will get more and more complex.

I don't think it's quite that bad. The design of ExclusiveReader is largely meant to just wrap the logic already present in Readable(Byte)Stream. There shouldn't be any duplication, at least in theory.

> When a user defines a custom stream, it should provide its own exclusive reader - Is it possible, given that we've had difficulty in doing that?

I agree this is a concern. Hopefully we can come up with a fairly straightforward template for doing so. Maybe @tyoshino's suggestions can help simplify the model in that direction.

@tyoshino 

> I recommend that we stop and think about what's needed for the WritableStream to add an ExclusiveStreamWriter to it. It would give us some insight about what states async reading should have for representing feeding aspect in order to support lots of requirements we have but without dragged by reading (result emitting) aspect (thinking about both at the same time leads to confusion).

Good idea, I agree.

> For example, "waiting" means that you cannot read() on ReadableStream but it doesn't disallow write() on WritableStream. If we add Writer to WritableStream, we cannot tell the user that they cannot call write() by returning state of "waiting". We should add "locked" state.

Hmm. Writable streams already have so many states. And the states there mean fairly different things from readable streams. So I am a bit reluctant. But maybe adding "locked" to both readable and writable streams would be a good simplification.

> Regarding the question I placed in #288 (comment), I now prefer (2) for release behavior. I've been feeling that one who called getReader() should be aware of that the stream is locked and should wait until the reader is released. Allowing something like (3) (not only for this point but for any other stuff such as that .closed can be used for getting notified of transition to "close") are huge source of complexity and doesn't benefit user so much, I guess.

Yeah, I think I agree now---having to try to implement even the closed getter does just make things annoying and complicated.

I am not 100% sure that a "locked" state will be better. But it is worth trying.

> I'd just add get released() and "locked" state so that the code that called getReader() and passed the reader to someone else can wait until the lock is released.

Where would you put released---on the reader, or on the stream? I think having it on the reader is pretty OK. For the stream I would like to make ready or wait() (or pull()) notify of all state transitions, including from "locked" to some other unlocked state. But that might also be a source of complexity.

> Radically speaking, I think even get released() is unnecessary and the original caller who obtained the reader should ask the user of the reader to notify of completion of its processing manually.

Yeah. My mental model is generally that you should not pass readers around. Once you take a reader, you control the stream until you release it. And if you call a function (like pipeTo) that you expect to exclusively lock the stream, you should probably stop touching the stream until that function signals that it's done---e.g. by returning a promise, like pipeTo does.

---

What I am mainly getting from this is a few things:

- Both here and in #289, we need to make sure the whole system works together. That should ideally include readable streams, writable streams, exclusive readers, exclusive writers, plus byte versions of all four of those. To validate our ideas here I think it will largely suffice to write examples, but they will have to be in-depth examples that consider a variety of underlying sources and sinks and use cases.
- We need to carefully consider how we can simplify the complexity of ExclusiveReader. It should not require the level of hard work we've had to put in to designing it. As @yutakahirano reminds me, we took a lot of time to get something for ReadableStream, and it was fairly complex; I got stuck trying to do it for async ReadableStream; and we haven't even thought about it yet for ReadableByteStream. It needs to be easier, both for us and for anyone who creates custom streams. One proposal is the "locked" state which might simplify things a lot. To validate our ideas here we'll need to prototype them.
- We need to figure all this out fast. Chrome would like to ship fetch + streams soon, and Igalia is implementing ReadableStream in WebKit, and Firefox is looking into it. I have faith that we can find the right solution, but we can't keep going back and forth too much. So my approach here is to try to evolve existing stuff, since we have worked out a lot of the bugs in the existing design already.


---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/pull/288#issuecomment-76329244