Re: [XHR] chunked

On Wed, Nov 30, 2011 at 7:28 AM, Anne van Kesteren <annevk@opera.com> wrote:
> A while ago sicking proposed adding chunked support to XMLHttpRequest:
>
> http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0741.html
> https://bugzilla.mozilla.org/show_bug.cgi?id=687087
>
> A use case I remember was downloading a large file of some kind that
> presumably can be incrementally rendered as otherwise responseType "blob"
> should be sufficient. More use cases appreciated. Would help with the
> design.

Incremental rendering as well as incremental parsing. Imagine for
example a webapp like google docs but with the ability to read MS-Word
documents directly client-side (currently I believe google docs does
their MS-Word file parsing server-side).

Even if the application wouldn't want to render the document right
away, parsing it incrementally rather than after the full download has
several advantages. First of all, external resources referenced from
the document can be loaded immediately and in parallel with the main
document. Second not having to keep the unparsed data of the full
document can save a bunch of memory. Third, it means that the document
is available for use sooner after the download is finished since most
if it is already parsed.

> Charles asked whether "chunked-text" was really needed (and whether we
> should have "chunked" which implies ArrayBuffer instead). Nobody got back to
> him on that.

Any text based format would benefit from chunked-text. While the
example above uses a binary format, it applies equally to text based
formats. And given how much we in this group seem to prefer text based
formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should
assume that other people at least use them, if not prefer them.

> If it is needed, how does it work when you just have some of
> the bytes of a multi-byte character in a single chunk? Fails to decode as
> per the normal algorithm?

Any multi-byte characters which are only partially downloaded are not
returned in that "chunk". I.e. you only expose fully decoded
characters. The value in progressEvent.loaded doesn't match up with
the accumulated xhr.response.length for multi-byte encodings anyway.

> Also, this basically makes it possible to write EventSource on top of
> XMLHttpRequest. Is that acceptable? If it encourages more people to use a
> lower-level API, higher-level optimizations for mobile phones might become
> harder down the road.

It seems more than acceptable to me! It's a good sign if our low-level
primitives, like XHR, are extended to the point that they can be used
to implement our higher level APIs. That shows that we're building our
low-level primitives correctly.

And note that EventSource is already implementable on top of XHR. The
only thing that "chunked" is buying us is less memory usage.

But I also think that EventSource still has some significant
advantages in that it supplies things like automatic reconnection and
possibly in the future events delivered over SMS or other "push"
mechanisms.

/ Jonas

Received on Wednesday, 30 November 2011 19:33:44 UTC