Re: [streams] What types does ReadableByteStream's reader's read(x) accept and return? (#295)

I see. This is a tough one. Hmm.

> Then, in what data type should the view passed to the underlying byte source? As-is? Maybe ok. It's the underlying byte source's responsibility to fill the view passed through the pullInto interface and call controller.respond() with the same TypedArray with an adjusted viewed range.

Hmm. At first I thought as-is would be reasonable. But I am not sure underlying sources should have to care about this---it would probably be inconvenient for them, and they would always just unwrap it into an ArrayBuffer and/or a Uint8Array or DataView. Maybe underlying sources only get Uint8Arrays? See below.

> I think we shouldn't do this. E.g. TCP socket may deliver chunks cut at arbitrary points, even if the server sends aligned data.

Agreed.

---

Here are three major possibilities:

1. Require Uint8Array usage everywhere. Don't even accept other types as arguments. (I guess we could allow DataView... meh.) This puts the burden on the user. They have to say "hmm, I got back 2 bytes; that is not enough for me to extract and create a Float64 array, so I will call read() again using a view that is 2 bytes further into the backing ArrayBuffer."

 - Pro: gives control back to the user ASAP, without any long trips into I/O land.
 - Pro: will never need to error due to misaligned bytes, even at EOF.
 - Con: annoying for users who have to assemble larger structures more manually. (Especially since their ArrayBuffer objects will keep getting detached, and they have to retrieve new ones via `returnedView.buffer`.)

2. Do our best to always fill the entire supplied view. So let's say in your case we have: `controller.enqueue(new Uint8Array([1, 2]))`, then `const view = new Float64Array(2); rbs.read(view)`. The implementation then copies the two bytes (`1` and `2`) into the passed view, then calls `ubs.pullInto(new Uint8Array(view.buffer, 2, 14))`. If pullInto responds with fewer than 14 bytes, it keeps calling pullInto with the appropriate range, until eventually all 16 total bytes (2 filled from queue plus 14 filled from pullInto) are filled. Only then does the promise returned by `rbs.read(view)` fulfill.

  - Pro: less annoying for users. Note how the only time you will get back fewer bytes than you requested is EOF.
  - Con: does more work behind the scenes, instead of giving full low-level control. A single call to read() could cause multiple I/O operations.
  - Con: if we get a request for a Float64Array but there are fewer than 8 bytes left in the file, we have to error (or do something weird like return a Uint8Array instead).

3. Do our best to align to BYTES_PER_ELEMENT. Similar to (2), but we would go for 6 more bytes instead of 14 in that case.

  - Pro: if the user passes in a single-byte view, they get full control, and will not cause multiple I/O operations (obviating the above Con).
  - Con: if the users passes in a non-single-byte view, this could do more work behind the scenes, instead of giving full low-level control. A single call to read() could cause multiple I/O operations. (But, not as much as (2) would.)
  - Con: a bit annoying for users who really want to read a certain number of bytes.
  - Con: if we get a request for a Float64Array but there are fewer than 8 bytes left in the file, we have to error (or do something weird like return a Uint8Array instead).

---

What do you think of this analysis? It sounds like https://github.com/whatwg/streams/issues/295#issuecomment-108732963 is leaning more toward 2 or 3. Both seem kind of nice, although I am worried about how they are brittle in the face of EOF. I also imagine that they add a lot more complication than 1 does.

Maybe 1 is good enough?

---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/295#issuecomment-108980522