Re: Media Queries and optimizing what data gets transferred

On Sun, Jan 27, 2013 at 1:38 AM, Ilya Grigorik <ilya@igvita.com> wrote:
> (1) It is *not* a question either/or. To the extent possible, both a
> client-side and a server-side solution should be available for content
> adaptation. Posing Client-Hints as "breaking" or impeding client-side
> adaption misses the point entirely.

I disagree. It doesn't make sense to provide the facilities for
server-side adaptation if providing facilities for client-side
adaptation can yield better results. For example, I think we should
not have provided an Accept-Codecs mechanism in addition to the
multi-<source> client-driven alternative selection in <video>. In
particular, when the client chooses from a set of alternatives offer
to it, where each alternative has a distinct URL, the result is
maximally friendly to intermediate caches including CDNs. In the
multi-<source> design, videos can be on a dumb CDN that neither needs
to implement codec negotiation logic nor needs to be able to consult
an origin server for such negotiation. The CDN only needs to be able
to serve files in the setting where there is a single file of bytes
corresponding to each URL.

> (2) Some things are far better handled by the server, not by the client.
> Things like resizing images to optimal width and height, in the world of
> ever exploding form factors is easily automated at the server level, and
> leads to massive bloat on the client. I don't mean to pick on picturefill,
> but just for the sake of an example:
> https://github.com/scottjehl/picturefill#hd-media-queries

Does anyone with a popular site (but not as popular as Google)
actually want to rescale images on the fly (as opposed to letting
Opera do it for them)? I really doubt that on-the-fly scaling to
precise client dimensions would become popular across the Web.
Instead, I expect sites to generate handful of pre-scaled image files
and then arrange something for choosing between them. In such a
scenario where the set of pre-scaled images is known in advance, the
design trade-offs become similar to the <video> codec negotiation
case. If each scaled image has its distinct URL, the site can declare
all the available scaled images in HTML and let the browser choose.
And this will again work efficiently without any logic in CDNs.

As for bloat in terms of the number of URLs that need to be offered to
the client, I don't expect the number of different pre-scaled images
to keep growing and growing. At some point, sites will just opt to
letting some devices scale down a slightly larger image or scale up a
slightly smaller image. Also, the example you are referring to tries
to handle art direction and mere scaling in the same example. Of
course, you get more combinations when you do art direction. Without
art direction, you could use the same image in both the X width retina
and the X*2 non-retina cases.

I think the example looks complex, because it tries to shoehorn both
HiDPI support and width-based art direction in the same syntax. I
don't think it follows that an HT TP-based solution is needed. It
might be that a better solution would be separating the syntaxes for
HiDPI and art direction.

> The above only covers 3 breakpoints and HiDPI. Do you really expect everyone
> will start filling their pages with that or similar markup for *every*
> image asset on the page?

No. In the immediate future, I expect most sites to continue to serve
non-HiDPI images only. In the further future, I expect most sites
neither to do art direction nor to provide non-HiDPI images separately
once HiDPI becomes popular enough.

And in cases where people will bother to provide non-HiDPI and HiDPI
images as different files, I expect most sites not to bother to do art
direction for all their images. (Simply because art direction requires
human attention but a CMS can generate non-HiDPI automatically given
HiDPI input.)

So I think it's misleading to suggest that the syntax for the most
complex case would occur with every image asset.

> Having said that, images is just one example. Client-Hints is a generic,
> cache-friendly transport for client-server negotiation.

Why do you characterize Client-Hints as cache-friendly? It seems to me
that with Vary: Client-Hints, even the local cache gets invalidated if
the user rotates the device 90° or if the current bandwidth and
estimates changes.

>> Reasons why I think introducing an HTTP-based solution would be worse
>> than adjusting CSS include:
>>  * HTTP caches don't know what the negotiation logic is, so they need
>> to check with the origin server for each Client-Hints header value
>> that they don't already have a cache key.
>
> Not true. The whole point of Client-Hints is to enable caches to perform
> "Vary: Client-Hints". What you've described is how the process works
> today... the requests are forced down to origin because we don't have a
> clean cache key to vary on.

So if the cache knows that the response from the origin server varies
depending on what value the request header Client-Hints has, the cache
won't be able to use the cached response when a new request comes with
a different Client-Hints value—even from the same browser after its
bandwidth estimate has changed slightly—without consulting with the
origin server.

I don't see how Vary: Client-Hints makes Client-Hints cache-friendly,
when you can expect pretty much every request to have a different
Cache-Hints value.

>>  * If the origin server doesn't get ETags right, intermediate caches
>> end up having a distinct copy of the data for each distinct
>> Client-Hints header value even if there is a smaller number of
>> different data alternatives on the origin server.
>
> Etags has *nothing* to do with this, and ETags is also not a mechanism to
> vary different responses to begin with.

Have I misunderstood how HTTP cache validation works? If the cache
already has a response entity with an ETag and Vary: Client-Hints and
a new response to the cache comes in with a different value for
Client-Hints, isn't the cache supposed to issue a conditional request
with the ETag back to the origin server so that the origin server gets
to indicate whether the new Client-Hints value results in a different
response body or in the same one the cache already has?

>>  * Sending any HTTP header incurs extra traffic for all the sites that
>> don't pay attention to Client-Hints. That would be the whole Web at
>> least at first. That is, an HTTP-based solution involves a negative
>> externality for non-participating sites.
>
>
> This is easily addressed by making it an opt-in mechanism for HTTP 1.1.

How would you handle the initial contact with the site? How would
opting into Client-Hints be better than setting a cookie? You
mentioned that cookies don't work cross-origin. How would Client-Hints
opt ins work cross origin?

> Further, the "cost" of upstream bytes, which is in the dozens of bytes, is
> easily offset by saving hundreds of kilobytes in the downstream (in case of
> images). The order of magnitude delta difference is well worth it.

That might be true if you consider the cost in the context of a site
that actually pays attention to Client-Hints. But there's the
externality that you end up sending Client-Hints even to sites that
don't pay attention to it (unless it's opt in).

>>  * It's bad to have to add server-side logic when CSS almost has the
>> feature authors want but not exactly.
>
>
> Not true. From first hand experience with PageSpeed, we know that despite
> continuous reminders web authors are simply not optimizing their images
> correctly: wrong formats, wrong sizes, etc. Automation solves this problem.
> While not 100% related to this discussion, see my post here:
> http://www.igvita.com/2012/12/18/deploying-new-image-formats-on-the-web/

This kind of "server automation will save us" argument would be easier
to buy if someone had already demonstrated a Web server that
automatically runs pngcrush on all PNG files and compresses JPEGs with
a better encoder than the one in libjpeg.

Why isn't such a server in the popular use and why should we expect a
server that automatically scales images in response to Client-Hints to
enter into popular use?

Note that arguments from automation seem to preclude art direction use
cases, so it doesn't make sense to me to make the argument that
automation solves the problem while at the same time presenting a
syntax example that involves art direction as that syntactically
bloated solution.

>>  * It's conceptually bad in terms of the learnability of the Web
>> Platform that a slight adjustment to the desired behavior would
>> involve changing the solution the author needs to apply to a
>> completely different technology in the stack (from MQ to HTTP
>> negotiation).
>
> Once again, they're not exclusive. If you don't have a server that can
> support image optimization, you should be able to hand-tune your markup. I'm
> all for that.

"Not exclusive" means that there's more stuff—hence bad for
learnability of the platform.

-- 
Henri Sivonen
hsivonen@iki.fi
http://hsivonen.iki.fi/

Received on Monday, 28 January 2013 13:24:25 UTC