Re: [whatwg/streams] light weight transformations (#461)

@ariutta pull-streams can be sync.

since your earlier post here, I've been thinking about what the ideal lowest common denomenator is.
in the line parser (split) you need to return `{remainder: '...', output: [...]}`, and then you flatmap the output,

but pass the remainder back into the input the next time. You _could_ use reduce for this, but then you have to pass back the whole `{remainder, output}` object, and it just appends to the output array.
reduce works, but if you need to parse an enormous file, you actually only need a tiny bit of state.
a stream that moves slowly, and where you want data out in real time is the same. With reducers, can you get data out before the stream has ended?

I've been thinking maybe there is a lower level, which reduce, and split are both special cases.
something like, you can return a feedback state, and an output so far. if the parser returns the state, instead of encapsulating it then it's still functional, but also very lightweight. 

other examples of operations which are more like `split` are `unique` (immediately emit the first time you see an item, but not again) or things that calculate something based on a sliding (etc) window

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/461#issuecomment-237062145