Re: [whatwg/streams] light weight transformations (#461)

> @domenic: How would back pressure propagation work in this scheme? If enqueue() does not return a promise, would there be another interface to pace the transform?

I'm not sure I quite understand the question. It would return a promise, so that's a pretty solid way to pace the transform. So `rs` would automatically get backpressure applied.

> Also, if it does not return a promise & concurrent calls to enqueue() are supported, buffering between stages of a pipeline becomes mandatory to ensure in-order processing. While the reference ReadableStream implementation is certainly not the last word on buffering performance, it seems very likely that such buffering is going to be more expensive than Promise.then().

I'm not sure what you mean by "concurrent" here, since JS doesn't have concurrency except with web workers. Do you mean the transform enqueuing multiple chunks in response to one incoming chunk? Yes, that is definitely supported. But in `rs.pipeThrough({ writable, readable })`, `readable` already has a buffer we can use---one which is automatically drained if there are outstanding read requests (which is I think what you are referring to by talking about `Promise.prototype.then()`). So I guess the cost you are worried about is about putting an element in the array, then taking it out? I'm not really concerned about that, and it can be optimized away if necessary to just directly resolve the pending promise.

---

I noticed later people are talking about flatMap. In case it isn't clear, that's exactly the existing transform() interface, just using enqueue() instead of returning an array.

---

@ariutta

> but the smallest I've been able to get the web streams polyfill down to was 47K minified.

Well, this is because of two factors. One, you're copying the reference implementation. A from-scratch polyfill would be a lot smaller, inlining functions and removing asserts and such. Two, you're using a transpiler, and those are well-known for massive bloat in output size. If you need ES5 or ES3 compatibility, you really need to write the code in ES5/3, instead of transpiling from ES6.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/461#issuecomment-237778461