Re: [streams] How will ReadableByteStream work with seekable resources? (#253)

> Hmm, but now I wonder if this is something we should define in the Streams standard or not. Maybe we should just include some advice how to encapsulate blocking I/Os with the ReadableStream interface?

What is really worrying to me is that we seem to have painted ourselves into a corner, so that now all the ideas that we're considering involve passing off the resulting complexity to authors. Whether it be `feedArrayBuffer()` or `setAllocator()` or "follow these guidelines in the spec," in all cases it feels like this kind of sucks. All of the stuff we're discussing now seems too low-level---it's more stuff that the stream creator should be concerned with, when developing their underlying source object, and not stuff the consumer should be concerned with.

Stated another way. One of the things the current design does really well, I think, is focus on unifying push and pull sources into a single author-facing API: a "readable stream." I think it should be within our charter to unify read(2)-backed and fread-backed readable byte streams. (And, I also think it's important to make readable byte streams a special case of readable streams---or an extension of them, depending on how you look at it.)

At this point I really don't see any truly *good* solution except moving to async read---everywhere. But I feel really bad about this. One of the very original points of contention between your W3C streams draft and my WHATWG streams draft, inspired by Node streams, was saying that sync read was important. Now, I don't think at the time either of us had a complete understanding of all the issues involved, and how it would impact the byte-stream case due to this fread vs. read(2) dichotomy combined with the desire to specify up front how many bytes you want to read. (Or maybe you did and I just wasn't listening? :-/) We've learned a lot about the problem domain since and in general it's good to revise APIs in light of new information. But it's late! You guys have an implementation in Chrome that you want to ship soon! And making this big of a change this late gives off instability signals that make me quite sad.

I guess what I'm asking is---if I can draft up complete semantics for a switch to async read over the next week, how disruptive do you think this would be? To the implementation you guys are hoping to ship soon for Fetch, and to the ecosystem overall? Here's a tentative sketch of what I'm thinking, happy to revise:

### ReadableStream

- `ReadableStream.prototype.read()` becomes promise-returning
  - It fulfills with `ReadableStream.EOS` ("end of stream") when there is no more data
- `ReadableStream.prototype.state` combines `"readable"` and `"waiting"` into a single state, probably named `"readable"`.
- `ReadableStream.prototype.ready` disappears
- Multiple calls end up asking for subsequent chunks, i.e. `rs.read().then(...); rs.read().then(...);` should pull two chunks out, instead of e.g. rejecting the second promise or fulfilling both with a single chunk.
  - Alternately: if the second call rejects, then we might be able to get rid of exclusive readers, since in that case if you do `rs.read().then(chunk => { rs.read().then(...); })`, i.e. if you immediately react to the read promise fulfilling by reading from it again, then I don't think anyone else can interfere.

### ReadableByteStream

- `ReadableByteStream.prototype.readInto(ab, offset, size)` becomes promise-returning, fulfilling with the number of bytes written into `ab`.
  - For fread-backed streams, it calls blocking `fread(ab, offset, size)` in another thread
  - For read(2)-backed streams, it can behave similarly to now, except that instead of writing zero bytes into `ab` it can block until there are bytes to write. (And maybe it should in general block until there are `size` bytes to write, except at end-of-stream?)

**Please let me know ASAP if you think I should work on this**. If you think it's too late for such a big change and we should explore other options, I can understand. But if you think it's worth doing then I want to get it ready ASAP so as to minimize disruption.

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/253#issuecomment-74114680