Re: [streams] Operation stream (#287)

Domenic: Response to https://github.com/whatwg/streams/pull/287#issuecomment-76333828 cont'd

> These are really interesting observations. Can you explain more why a readable stream would need acknowledgement/error for each chunk read?

When we were discussing the idea of having a buffer pool inside a readable stream and giving them to the user on reading, we found that we need to explicitly tell the source when the buffer becomes ok to reuse. As you said, `ReadableStream`'s `read()` implicitly acknowledges the data, it's not always that we finish consuming contents on the read buffer synchronously. So, I attempted to add explicit and delay-able acknowledgement feature as that read() returns an object on which methods for acknowledgement are placed.

```
const dataAndAckMethod = s.read();
// Use data part.
...
// When data is comsumed (possibly in another task), call the ack method
```

In addition to that, I thought that similarly to bring-your-own-buffer model readable stream, we may want to use please-write-to-this-buffer model writable stream. I.e. suppose that we have a sink that prepares and exposes some memory region as an ArrayBuffer and ask the producer code to write bytes into it. To wrap this sink with some streams variant API, we need a method which returns a region for writing data and also returns some methods to tell the sink completion of the writing.

```
const regionAndCompleteMethod = s.write();
// Write data to the region
...
// When done (possibly in another task), call the complete method.
```

This reduces the number of ArrayBuffer allocation compared to please-give-me-a-filled-array-buffer model writable stream. This `s.write()` is logically equivalent to current WHATWG ReadableStream if the return value type is changed to the Operation object.

----

Anyway they're just attempts to address each of the semantics separately. Now, I'd like to go back to investigate which (set of) semantics we should support

Source of bytes

- Choice: who allocates buffers
  - Code prepares and passes an ArrayBuffer to source. Source writes bytes to it and notifies code of completion.
    - writing bytes completes synchronously (when read() returns, writing is finished).
    - Source notifies code of completion asynchronously.
  - Source prepares and passes a filled ArrayBuffer to code.
    - Code returns the ArrayBuffer to source when done consuming.
    - GC collects consumed ArrayBuffers.

Sink of bytes

- Choice: when write is allowed
  - Sink automatically starts accepting write (w/ backpressure hint)
  - Some preceding method call makes sink accept write
- Choice: who allocates buffers
  - Sink prepares and passes an ArrayBuffer to code. Code writes bytes to it and notifies sink of completion.
    - Choice: allocation is async? sync?
      - write method returns an ArrayBuffer synchronously
      - write method returns an ArrayBuffer asynchronously
  - Code prepares and passes a filled ArrayBuffer to sink.
    - Choice: recycle the buffer?
      - Sink consumes contents synchronously, so code can start reusing the buffer immediately.
      - Sink notifies code once I finish consuming bytes from it to allow code to recycle it.
      - Sink takes ownership of the ArrayBuffer and have the GC collect it once consumed.


---
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/pull/287#issuecomment-76695491