Re: #445: Transfer-codings

On 2014–04–07, at 1:29 PM, Matthew Kerwin <matthew@kerwin.net.au> wrote:

> That's a big value judgement on what constitutes good site design. Yes, in lots of cases it makes sense to compress your resources and have multiple representations, especially for static resources; but what about the sites that aren't like that? Why is it bad site design to have a big resource that can be accessed with ranges? If the answer is because doing so would require TE in order to have compression, then it's a tautology (TE is only needed by bad sites; those sites are bad because they need TE). If it's because caches don't handle that properly, then it's a chicken-and-egg problem. The only reasons I can think of for calling them bad are either circular, or "I don't like them." Is there a real reason?

Range requests don’t get cached because they are uncommon (not chicken and egg, but because there are few potential applications) and it greatly complicates cache design. The underlying resource is likely to be too large to be stored entirely by the client, so the cache would have to store ranges separately and coalesce them on demand, per query. A single cache miss potentially generates multiple sub-range requests on the network.

Custom, application-specific caching, which may know about specific usage patterns or server performance characteristics, looks better.

> Why should I make my web API use "?start=N&end=M" when I could use "Range: x-records=N-M” ?

Why not accept both? If no cache handles range requests, at least not differently from the query string, there is nothing to gain by being futuristic. You can use your extension when it’s supported, but note the high implementation cost along the entire transport chain. Is it worth it?

A simple adaptation, which surely isn’t my original invention just now, is to store large sub-ranges as virtual files. For example, go by powers of two, so you can download any two megabytes starting at any offset multiple of 2^21, any 64 kilobytes starting at a multiple of 2^16, etc. These are cache-friendly and simple to merge in client-side logic, and they work with content-encoding.