Re: Thoughts behind the Streams API ED

stop/resume:

Indeed as I mentioned this is related to WebCrypto Issue22 but I don't 
think this is a unique case. Issue22 was closed because of lack of 
proposals to solve it, apparently I was the only one to care about it 
(but I saw recently some other messages that seem to be related), and 
finally this would involve a public clone method with associated 
security concerns.

But with Streams it could be different, the application will internally 
clone the state of the operation probably eliminating the security 
issues, as simple as that.

To describe simply the use case, let's take a progressive hash computing 
4 bytes by 4 bytes:

incoming stream: ABCDE bytes
hash operation: process ABCD, keep E for the next computation
incoming stream: FGHI bytes + STOP-EOF
hash operation: process EFGH, process STOP-EOF: clone the state of the 
hash, close the operation: digest hash with I

resume:
incoming stream: JKLF
hash operation (clone): process IJKL, keep F for next computation
etc...

So you do not restart the operation as if it was the first time it was 
receiving data, you just continue it from the state it was when stop was 
received.

That's not so unusual to do this, it has been requested many times in node.

pipe/fork: I think b is better.

Regards

Aymeric



Le 06/11/2013 20:15, Takeshi Yoshino a écrit :
> On Wed, Nov 6, 2013 at 7:33 PM, Aymeric Vitte <vitteaymeric@gmail.com 
> <mailto:vitteaymeric@gmail.com>> wrote:
>
>     I have seen the different bugs too, some comments:
>
>     - maybe I have missed some explaination or some obvious thing but
>     I don't understand very well right now the difference/use between
>     readable/writablebytestream and bytestream
>
>
> ReadableByteStream and WritableByteStream are defining interfaces not 
> only for ByteStream but more generally for other APIs. For example, we 
> discussed how WebCrypto's encryption method should work with Stream 
> concept recently, and one idea you showed was making WebCrypto.subtle 
> return an object (which I called "filter") to which we can pipe data. 
> By defining a protocol how to pass data to consumer as the 
> WritableByteStream interface, we can reuse it later for defining IDL 
> for those filters. Similarly, ReadableByteStream can provide uniform 
> protocol how data producing APIs should communicate with consumers.
>
> ByteStream is now a class inheriting both ReadableByteStream and 
> WritableByteStream (sorry, I forgot to include inheritance info in the 
> IDL).
>
>     - pause/unpause: as far as I understand the whatwg spec does not
>     recommend it but I don't understand the reasons. As I previously
>     mentionned the idea is to INSERT a pause signal in the stream, you
>     can not control the stream and therefore know when you are pausing it.
>
>
> Maybe after decoupling the interface, pause/unpause are things to be 
> added to ByteStream? IIUC, pause prevents data from being read from a 
> ByteStream, and unpause removes the dam?
>
>     - stop/resume: same, see my previous post, the difference is that
>     the consuming API should clone the state of the operation and
>     close the current operation as if eof was received, then restart
>     from the clone on resume
>
>
> Sorry that I haven't replied to your one.
>
> Your post about those methods: 
> http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0343.html
> WebCrypto ISSUE-22: http://www.w3.org/2012/webcrypto/track/issues/22
>
> Maybe I still don't quite understand your ideas. Let me confirm.
>
> stop() tells the consumer API implementing WritableByteStream that it 
> should behave as if it received EOF, but when resume() is called, 
> restart processing the data written between stop() call and resume() 
> call as if the API received data for the first time?
>
> How should stop() work for ByteStream? ByteStream's read() method will 
> receive EOF at least once when all data written before stop() call has 
> been read, and it keeps returning EOF until resume() tells the 
> ByteStream to restart outputting?
>
> I've been feeling that your use case is very specific to WebCrypto. 
> Saving state and restoring it sounds more like feature request for 
> WebCrypto, not a Stream.
>
> But I'm a bit interested in what your stop()/resume() enables. With 
> this feature, ByteStream becomes message stream which is convenient 
> for handling WebSocket.
>
>     - pipe []/fork: I don't see why the fast stream should wait for
>     the slow one, so maybe the stream is forked and pause can be used
>     for the slow one
>
>
> There could be apps that want to limit memory usage strictly. We think 
> there're two strategies fork() can take.
> a) wait until the slowest substream consumes
> b) grow not to block the fastest substream while keeping data for the 
> slowest
>
> a) is useful to limit memory usage. b) is more performance oriented.
>
>
>     - flow control: could it be possible to advertise a maximum
>     bandwidth rate for a stream?
>
>
> It's currently communicated as window similar to TCP. Consumer can 
> adjust size argument of read() and frequency of read() call to match 
> the consumer's processing speed.

-- 
Peersm : http://www.peersm.com
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms