Re: [streams] Support reading bytes into buffers allocated by user code on platforms where only async read is available (#253)

Summary of offline discussion with Yutaka.

(1) We did further investigation about when we want to do pipe. I now believe that we basically don't run pipe or any other operation requires locking until we finish reading results of pending reads. If we know the position from which we want to start piping (e.g. once header is read and body starts), we just feed sufficient necessary amount of Uint8Arrays and just wait for all views are read. If we don't know the position, we supply enough Uint8Arrays and if fed too much, wait for read completion and pass extra views to the operation which requires locking together with the stream.

I wrote "bad" in the table in https://github.com/whatwg/streams/pull/288#issuecomment-76885522, but now I think it's just fine to ask the user to fully consume the fed buffers and then start piping. We may ask the source to tell the stream whether there's any pending read or not and fail `getReader()` if yes.

----

(2) Again, I think the ReadableStream interface we have in the spec now is great. As Domenic should also notice, async read() is not great in terms of the following points:

- when there's any pending read, the user should watch at the promise returned by read(), while if there is not any pending read, the user should watch at the stream.
- we may want to feed small ArrayBuffer fragments at once. in such case, in the async read() approach, we obtains multiple promises though the stream is abstraction of a sequence. we're not running parallel computation. it seems a signal of bad design that we get multiple promises.
- for object streams, it doesn't make sense to have read() finish async

sync read() + state indicating readability + notification by a single promise

This looks the most ideal interface for reading sequence.

----

(3) Given (1), I think pull + sync read() approach is ok with locking infrastructure. And given (2), I'd like to pursue this path.

(4) Yutaka said in https://github.com/whatwg/streams/issues/253#issuecomment-76910547 that

> Otherwise, it is hard to define getReader. As I wrote below, I want to decouple pending read operations and the lock behavior, but I couldn't think of any reasonable rule determining if there is a pending read except for counting calls.

But he withdrew it after discussion.

----

(5) One of suggestions included in my operationStream branch is separation of queue and stream interface. Given this change, our proposal written in Yutaka's this comment https://github.com/whatwg/streams/pull/288#issuecomment-76178348 is worth being revisited. Now queue lives in the source. There's no worry about need for putting back chunks already read into ReadableStream.

Even if we place pipe on the source, https://github.com/whatwg/streams/pull/288#issuecomment-76193197 can be realized. We just obtain and pass a reader to the pipe algorithm so that they can implement pipe in JS.

We add createAutoStream() and createManualFeedStream() to the source. The user calls them to get a stream they want.

----

(6) The dual of this design for writable side is write() and readDisposed(). readDisposed() has the same semantics as ReadableStream.read(). From readDisposed(), we can read ready to reuse ArrayBuffers. The dual of auto mote is auto disposing writable stream where readDisposed() is unavailable. Used ArrayBuffers are detached from GC root and get collected.


---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/253#issuecomment-76958542