- From: Domenic Denicola <notifications@github.com>
- Date: Mon, 16 Mar 2015 03:10:37 -0700
- To: whatwg/streams <streams@noreply.github.com>
- Message-ID: <whatwg/streams/issues/294/81557090@github.com>
Tentative plan per afternoon discussions: - Use options object to allow greater future flexibility - Instead of "manualPull: false/true", use a `queueSize` argument, for both object and byte streams. If it is zero, then pulling is manual. (If `feedBuffers` is true, it must be zero.) If it is nonzero _n_, it communicates two things: - The consumer would prefer that, on a best-effort basis, the stream have _n_ bytes available to fulfill future .read() promises quickly. (This means for sources like files, the underlying source should pull enough bytes proactively to meet _n_.) If _n_ is 10 MiB, then that might mean a single .read() promise gets fulfilled with a 10 MiB chunk - The consumer would prefer not to consume much more memory than _n_. If the stream is getting close to or exceeding this threshold, then the stream should apply backpressure (or stop pulling proactively, for a case like files). - To make nonzero queueSize work, an underlying source must implement the pull()method---if that is not implemented, trying to pass queueSize will throw. - To make feedBuffer: true work, an underlying source must implement the read() method---if that is not implemented, trying to pass feedBuffers will throw. - Cases like a ReadableStream wrapping a `WebSocket` (i.e., wrapping an API that does not support backpressure signals) will not implement underlying source pull(), causing `queueSize` to never work. Instead a simple `.getReader()` call must be used, since we should not allow consumers to signal a desired queueSize if there is no way to control it. - Strategy will probably change. The most likely path looks to be getting rid of shouldApplyBackpressure and moving size() to the underlying source directly. - It would be beneficial to get the more precise backpressure APIs prototyped in #287, e.g. maybe enqueue() should return the size left in the queue or similar. We should work on this. - The implementation may get complicated in how we allow scenarios like ```js const reader = rbs.getReader({ queueSize: 1024 }); // later, after the queue has filled up reader.releaseLock(); const reader2 = rbs.getReader({ feedBuffers: true }); reader2.read(new Uint8Array(512)); ``` This kind of exotic scenario will probably necessitate copying (at least if implemented in JavaScript; UAs might be able to get away with better tricks for their internal queues). --- Reply to this email directly or view it on GitHub: https://github.com/whatwg/streams/issues/294#issuecomment-81557090
Received on Monday, 16 March 2015 10:11:06 UTC