Re: Overlap between StreamReader and FileReader

Le 13/09/2013 15:11, Takeshi Yoshino a écrit :
> On Fri, Sep 13, 2013 at 9:50 PM, Aymeric Vitte <vitteaymeric@gmail.com 
> <mailto:vitteaymeric@gmail.com>> wrote:
>
>
>     Le 13/09/2013 14:23, Takeshi Yoshino a écrit :
>>     Do you mean that those data producer APIs should be changed to
>>     provide read-by-delta-data, and manipulation of data by js code
>>     should happen there instead of at the output of Stream?
>
>     Yes, exactly, except if you/someone see another way of getting the
>     data inside the browser and turning the flow into a stream without
>     using these APIs.
>
>
> I agree that there're various states and things to handle for each of 
> the producer APIs, and it might be judicious not to convey such API 
> specific info/signal through Stream.
>
> I don't think it's bad to convert xhr.DONE to stream.close() manually 
> as in your example 
> http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0453.html.
>
> But, regarding flow control, as I said in the other mail just posted, 
> if we start thinking of flow control more seriously, maybe the right 
> approach is to have unified flow control method and the point to 
> define such a fine-grained flow control is Stream, not each API.

Maybe, I was not at the start of this thread too so I don't know exactly 
what was the original idea (and hope I am not screwing it up here). But 
I am not sure it's possible to define a universal flow control.

Example: I am currently experiencing some flow control issues for 
project [1], basically the sender reads a file AsArrayBuffer from 
indexedDB where it's stored as a Blob. Since we can not get delta data 
while reading the File for now, the sender waits for having the whole 
ArrayBuffer, then slices it, processes the blocks and sends them via 
WebSockets. If you implement a basic loop, of course you overload the 
sender's UA and connection. So the system makes some calculation in 
order to allow only half of the bandwidth to be used, aggregate the 
blocks until it finds out that the size of the aggregation meets the 
bandwidth requirement for the aggregated blocks to be sent every 50ms.

Then it uses a poor setTimeout to flush the data which screw up all the 
preceding calculations since setTimeout fires whenever it likes. Maybe 
there are smarter ways to do this, I was thinking to use workers so you 
can get a more precise clock via postMessages but I did not try.

In addition to the bandwidth control there is a window for flow control.

I take this example to understand if this could be better with a 
built-in Stream flow control, if so, after you have defined the right 
parameters (if possible) for the streams flow control, you could process 
delta data while reading the file and restream them directly via 
WebSockets, and this would be great but again not sure that a universal 
solution can be found.

> If we're not, yes, maybe your proposal (deltaResponse) should be enough.

What is sure is that delta data should be made available instead of 
incremental ones.

[1] http://www.peersm.com

-- 
jCore
Email :  avitte@jcore.fr
Peersm : http://www.peersm.com
iAnonym : http://www.ianonym.com
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms
Web :    www.jcore.fr
Extract Widget Mobile : www.extractwidget.com
BlimpMe! : www.blimpme.com