Re: Thoughts behind the Streams API ED

On Wed, Nov 6, 2013 at 7:33 PM, Aymeric Vitte <vitteaymeric@gmail.com>wrote:

>  I have seen the different bugs too, some comments:
>
> - maybe I have missed some explaination or some obvious thing but I don't
> understand very well right now the difference/use between
> readable/writablebytestream and bytestream
>

ReadableByteStream and WritableByteStream are defining interfaces not only
for ByteStream but more generally for other APIs. For example, we discussed
how WebCrypto's encryption method should work with Stream concept recently,
and one idea you showed was making WebCrypto.subtle return an object (which
I called "filter") to which we can pipe data. By defining a protocol how to
pass data to consumer as the WritableByteStream interface, we can reuse it
later for defining IDL for those filters. Similarly, ReadableByteStream can
provide uniform protocol how data producing APIs should communicate with
consumers.

ByteStream is now a class inheriting both ReadableByteStream and
WritableByteStream (sorry, I forgot to include inheritance info in the IDL).


> - pause/unpause: as far as I understand the whatwg spec does not recommend
> it but I don't understand the reasons. As I previously mentionned the idea
> is to INSERT a pause signal in the stream, you can not control the stream
> and therefore know when you are pausing it.
>
>
Maybe after decoupling the interface, pause/unpause are things to be added
to ByteStream? IIUC, pause prevents data from being read from a ByteStream,
and unpause removes the dam?


> - stop/resume: same, see my previous post, the difference is that the
> consuming API should clone the state of the operation and close the current
> operation as if eof was received, then restart from the clone on resume
>

Sorry that I haven't replied to your one.

Your post about those methods:
http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0343.html
WebCrypto ISSUE-22: http://www.w3.org/2012/webcrypto/track/issues/22

Maybe I still don't quite understand your ideas. Let me confirm.

stop() tells the consumer API implementing WritableByteStream that it
should behave as if it received EOF, but when resume() is called, restart
processing the data written between stop() call and resume() call as if the
API received data for the first time?

How should stop() work for ByteStream? ByteStream's read() method will
receive EOF at least once when all data written before stop() call has been
read, and it keeps returning EOF until resume() tells the ByteStream to
restart outputting?

I've been feeling that your use case is very specific to WebCrypto. Saving
state and restoring it sounds more like feature request for WebCrypto, not
a Stream.

But I'm a bit interested in what your stop()/resume() enables. With this
feature, ByteStream becomes message stream which is convenient for handling
WebSocket.


>  - pipe []/fork: I don't see why the fast stream should wait for the slow
> one, so maybe the stream is forked and pause can be used for the slow one
>

There could be apps that want to limit memory usage strictly. We think
there're two strategies fork() can take.
a) wait until the slowest substream consumes
b) grow not to block the fastest substream while keeping data for the
slowest

a) is useful to limit memory usage. b) is more performance oriented.


>
> - flow control: could it be possible to advertise a maximum bandwidth rate
> for a stream?
>

It's currently communicated as window similar to TCP. Consumer can adjust
size argument of read() and frequency of read() call to match the
consumer's processing speed.