[whatwg/streams] Why does the `read` method of `ReadableStreamBYOBReader` class, need to _transfer_ the buffer it is provided? (Issue #1348)

amn created an issue (whatwg/streams#1348)

### What is the issue with the Streams Standard?

I only learned of the particular property of `read` very recently when I tried to amend a stream processing pipeline that relies on a WebAssembly routine to process a stream chunk by chunk, to eliminate [unnecessary] buffer copying. Originally I used the stream's iterator implicitly with a `for await` loop that vended `ArrayBuffer` objects (chunks), which had to be copied into the WASM instance memory (`memory`) buffer using e.g. `new Uint8Array(memory.buffer).set(new Uint8Array(chunk))`, the unnecessary copying part I then wanted to eliminate. I thought if I used a bring-your-own-buffer (BYOB) stream reader (`reader`) instead, I could instead "load" the WASM instance memory buffer directly from the stream.

Turns out, because buffers backing `WebAssembly.Memory` objects, are inherently _not transferrable_ (rooted probably in WASM designers wanting to allow for additional potential constraints in a given WASM implementation like guard pages to contain stack overflow errors and what not), a construct like `reader.read(new DataView(memory.buffer))` would invariably throw an error as it unsuccessfully attempts to detach the buffer backing the view it is supplied, in due accordance with step 10 in [the "Strems Standard" specification](https://streams.spec.whatwg.org/#readable-byte-stream-controller-pull-into).

This makes BYOB unusable with specifically WASM like that, which is IMO a great detriment as pairing two features that largely should help with performance bottlenecks, is currently a "no-go".

To that end, I'd like to ask -- why mandate _in the specification_ that the buffer be transferred (and thus must be transferable)? What also seems a bit "arbitrary" to me, everything considered, is that there's no clear reason why step 10 is even necessary per the spec. -- if it is omitted and `_bufferResult_.[[Value]]` is instead the actual buffer backing the view supplied, `_view_.[[ViewedArrayBuffer]]`, then the rest of the steps still appear perfectly valid, and since this is a _specification_, after all, it would have allowed for implementations to avoid transferring and thus detaching (effectively invalidating) the buffer provided to `read`. Buffers backing `WebAssembly.Memory` would be usable with `read`. There must be something I am missing with my interpretations here?

To add to my argument where I am trying to point at the requirement for "transferring" the buffer supplied, being a peculiarity that IMO could have been left out but isn't, we can consider the case of [the specification of] [`TypedArray.set`](https://tc39.es/ecma262/multipage/indexed-collections.html#sec-%typedarray%.prototype.set), for instance, as an example of a Web API that also does effectively load a [BYOB] buffer, but whose specification does not warrant a "transfer" (there is the parable 23.a using the term "transfer" but there's no indication the word refers to the same concept) and instead simply specifies a "loop over bytes and copy each". And so I've got to ask, in light of this -- why couldn't the specification of `read` for `ReadableStreamBYOBReader` have been written in the same way, so that we could "bring our own" WASM memory buffers without issues?

Below is a list of related discussions:

* https://github.com/wasm-bindgen/wasm-bindgen/discussions/3579, where the accepted answer appears to explain why BYOB works the way it does (transferring the buffer it is given) but does so specifically in context of Rust and `wasmbindgen`, and so doesn't necessarily explain BYOB design unless you either have a pair of Rust glasses on, which I don't, or unless you understand why Rust should have a say on how the Strems Standard should be defined

* https://github.com/WebAssembly/design/issues/1397, which, again, attempts to tackle the issue from "the WASM side", asking for rationale behind some of the decisions in WASM, but which may have stemmed in part because there's the relative inability to use WASM with e.g. BYOB; I, for one, don't think there's necessarily anything wrong with the way the "memory" concept was defined in WASM; instead, I ask why transferring of [BYOB] buffers is necessary to define on the level of specification, which effectively makes using [BYOB] with specifically WASM currently impossible (at least using `ReadableStreamBYOBReader`)

* https://github.com/WebAssembly/design/issues/1162, which is mostly relevant due to [the comment](https://github.com/WebAssembly/design/issues/1162#issuecomment-349437979) that sheds light on why specifically WASM buffers that back up memory, have to be the way they are

-- 
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/1348
You are receiving this because you are subscribed to this thread.

Message ID: <whatwg/streams/issues/1348@github.com>