Re: [XHR] support for streaming data

On Wed, Aug 17, 2011 at 3:22 PM, Chris Rogers <crogers@google.com> wrote:
>
>
> On Mon, Aug 8, 2011 at 5:13 PM, Jonas Sicking <jonas@sicking.cc> wrote:
>>
>> Hi All,
>>
>> XHR Level 2 does wonders for making XMLHttpRequest better. However
>> there is one problem that we have run into with "streaming" data.
>>
>> Using .responseType="text" you can read the contents of the data as
>> soon as it comes in, i.e. you don't have to wait for the "load" event
>> to read the data. However you can't do the same for .responseType =
>> "arraybuffer". This is with good reason as arraybuffers are immutable
>> in size and you can't really know how much data you'll receive up
>> front. (You can make guesses based on the "content-length" header, but
>> it's not always there, and isn't always reliable even when it's
>> there).
>>
>> Another problem is that even if the webpage is able use
>> .responseType="text" and consume the data in an incremental manner,
>> the UA still ends up accumulating and holding large chunks of data in
>> memory.
>>
>> So in other words, the use case is pages that are able to
>> incrementally consume binary and textual data and wanting to do so
>> without having to wait until the whole resource has been downloaded,
>> and do so without requiring the UA to hold large amount of data in
>> memory.
>>
>> To solve this I propose we add two new values to .responseType:
>> "streaming-text" and "streaming-arraybuffer".
>>
>> When .responseType is set to "streaming-text" then during each
>> "progress" event, .response would be set to the text that was received
>> since the last "progress" event (and since the "loadstart" event for
>> the first "progress" event).
>>
>> Same thing when .responseType is set to "streaming-arraybuffer". In
>> this case .response is set to an ArrayBuffer containing the data
>> received since the last "progress" event.
>>
>> There is one non-ideal thing with this solution. Once the last chunk
>> of data has arrived, at least Firefox doesn't fire a "progress" event,
>> but instead just a "load" event. This means that people will have to
>> consume data both from the "progress" event and from the "load" event.
>>
>> Another solution would to make sure to always fire a "progress" event
>> for the last data before firing the "load" event. I personally like
>> this approach more. There *might* even be reasons to do that to ensure
>> that pages create progress bars that reliably reach 100% etc.
>>
>> / Jonas
>
> Hi Jonas,
> I like the idea of getting access to the incrementally "streamed" data.  I
> think there are definitely some interesting use cases for it.
> Instead of adding a "streaming-" prefix for "streaming-text" and
> "streaming-arraybuffer", another approach would be to add a new attribute
> .responsePresentation (not a very good name) which could have values:
> "concatenated"
> "chunked"
> (or something like that)
> The default value for .responsePresentation would be "concatenated", but
> could be changed to "chunked" for the streaming behavior.
> I'm not necessarily in favor of this approach versus "streaming-text" and
> "streaming-arraybuffer", but just thought I'd bring it up.

Given how closely this is tied to .responseType, I think it's better
to have a single property rather than two. That will additionally give
us proper behavior as far as throwing if you try to change the mode
after data has already starting coming in. .responseType is defined to
throw if you try to set it after the readystatechange event for the
HEADERS_RECEIVED state. That is exactly what we want for the streaming
mode flag.

> Also, it would be good to get Ian's opinion about this since he's working on
> similar stuff with Web Sockets.

Indeed!

/ Jonas

Received on Thursday, 18 August 2011 00:02:18 UTC