- From: John Mellor <johnme@chromium.org>
- Date: Tue, 21 Aug 2012 20:59:09 +0100
- To: whatwg@whatwg.org, www-style@w3.org
- Cc: Mathew Marquis <mat@matmarquis.com>, "Tab Atkins Jr." <jackalmage@gmail.com>, Edward O'Connor <eoconnor@apple.com>
CSS image-set and HTML img srcset are getting their first implementations but both APIs have serious shortcomings. We should fix them before it's too late: 1. Neither is of any use for flexibly-sized images. 2. srcset isn't as smart/intuitive as image-set. 3. image-set is less flexible than srcset. I'll go through these in turn, in decreasing order of importance (sorry about the length, but this is a complex topic and I've tried to avoid ambiguity). (This email is cross-posted to whatwg and www-style, since this is of equal relevance to HTML and CSS). 1. Neither is of any use for flexibly-sized images. These APIs have been designed with fixed-size images in mind, and they work well for that use case. For example if you have a width:320px image, and you need to decide whether to load the 2x "retina" file or the standard file. However they stop working when the width (or even just min/max-width) is flexible, for example width:100%, which makes them almost useless for responsive web design, and not great even for basic tasks like adapting to different size mobile devices. I'll explain why: If you have the same image saved in a variety of scales on the server (e.g. 320.jpg, 640.jpg, 1280.jpg, and 2560.jpg, named after their widths), then you want the browser to load the one which has the same number of pixels of image data as the number of device pixels that the image is taking up on the display (assuming for now that bandwidth isn't a concern, and that you don't have to worry about the page being displayed at multiple zoom levels due to pinch zoom). For example if your width:100% image is currently stretched to 1280 CSS px, then you want to load 1280.jpg on 1x low DPI devices and 2560.jpg on 2x high DPI devices. But if it has shrunk to 320 CSS px, then you want to load 320.jpg on 1x low DPI devices and 640.jpg on 2x high DPI devices. Neither image-set nor srcset can cope with this simple fundamental task! Nor can they, as currently specified: while the browser knows (or can estimate, if layout hasn't yet happened) the number of device pixels the image is taking up, it doesn't know how many pixels of image data the srcset/image-set entries contain until it downloads them, since it knows their dppx but not their size. To put it another way, since the browser doesn't know the intrinsic size of the image until it downloads one of the files (and divides the image data size by its dppx value), they simply cannot know how much a flexibly-sized image is being stretched by, and so they can't use that information as input when deciding which image to download. [With srcset, it is possible to hack together a srcset definition that will load approximately the right image file by combining dppx and viewport width restrictions -- see sample code<http://jsbin.com/aganaz/8/edit?javascript,live> -- but this is excessively cumbersome for practical use.] A simple solution to this problem, would be to provide a way for authors to tell the browser in advance what the intrinsic size of a flexibly-sized image is, such that the browser can calculate how many pixels of image data each option contains, and hence which would be most appropriate. For example, one could add an initial term to the srcset/image-set, providing the intrinsic size of the image: <img srcset="320x120, 320.jpg 1x, 640.jpg 2x, 1280.jpg 4x, 2560.jpg 8x"> selector { background: image-set(320x120, "320.jpg" 1x, "640.jpg" 2x, "1280.jpg" 4x, "2560.jpg" 8x); } This would be equivalent to: <img srcset="2560x960, 320.jpg 0.125x, 640.jpg 0.25x, 1280.jpg 0.5x, 2560.jpg 1x"> selector { background: image-set(2560x960, "320.jpg" 0.125x, "640.jpg" 0.25x, "1280.jpg" 0.5x, "2560.jpg" 1x); } For each image, after discarding images that don't satisfy the viewport width/height criteria in the usual manner if it's a srcset, the browser would multiply the provided intrinsic size by the image's "x" value to calculate the number of pixels of image data available in each dimension, then it would pick the one that most appropriately matches the number of device pixels the image is actually taking up in each dimension (see 2. below for why I recommend "most appropriate" instead of "strictly pick the smallest image whose image width >= device pixel width"). This initial intrinsic width term would be optional, as it is unnecessary for fixed-size images, but would be strongly recommended for flexible-size images as without it the "x" values would select purely on dppx grounds, which as explained above isn't useful. In the occasional event that different images have different intrinsic sizes it would be possible to specify several intrinsic sizes, each of which would apply to the subsequent images (until the next intrinsic size). For example: <img id="logo" style="width:100%" srcset=" 320x320, square.jpg 1x 400w, square-hd.jpg 2x 400w, 320x50, flat-s.jpg 1x, flat-m.jpg 2x, flat-l.jpg 4x, flat-xl.jpg 8x "> would show a square logo on portrait phones, but a more discreet flat logo on wider devices (in both cases using the file of appropriate dpi taking into account the width at which it is displayed). Remember that specifying the intrinsic size would only be needed for flexible-size images (including fixed-size images with flexible min/max-width/height), and that multiple sizes would only be necessary when using images of different intrinsic sizes (i.e. different aspect ratios and/or different intrinsic scale). An alternative notation might be to just directly specify the image data dimensions of each image, but that seems slightly more cumbersome: <img srcset="320.jpg 320x120, 640.jpg 640x240, 1280.jpg 1280x480, 2560.jpg 2560x960"> The redundancy could be reduced by providing only widths, or only heights, though it might get confusing to have a bunch of raw widths or heights mixed in with viewport w/h restrictions, so I think the syntax suggested earlier is preferable. 2. srcset not as smart/intuitive as image-set. The syntax for HTML img srcset and CSS image-set is very similar, thus developers will assume they behave the same. This would be great, except that they actually behave rather differently. For example, compare: <img srcset="low.jpg 1x, high.jpg 2x"> selector { background: image-set("low.jpg" 1x, "high.jpg" 2x); } Surely such a simple case must behave the same? But actually, if you view this on a device whose dppx is 1.1, the image-set will most likely load low.jpg, as it is the closest match for the display, while the srcset will load high.jpg, because the specified algorithm<http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content-1.html#processing-the-image-candidates> treats the dppx values as strict maximums. The ideal solution to this is to relax the specified algorithm for srcset to just say that the UA will pick the most appropriate image based on their declared characteristics, as specified for image-set<http://dev.w3.org/csswg/css4-images/#image-set-notation>. For example 2x would just mean that the image has a dppx of 2, and hence is probably a good match for devices whose dppx is close to 2. For consistency one might be tempted to make a similar adjustment for the w/h viewport size constraints (i.e. they would declare the intended viewport size rather than the maximum viewport size), however that seems less useful, since there are good reasons why an author might want to precisely define the viewport size cut-off at which the switchover occurs (unlike dppx, where the browser usually knows better, since it has insight into bandwidth, latency, financial cost per byte, etc). 3. image-set is less flexible than srcset. As mentioned above, since srcset and image-set have such similar syntax and semantics, it's important for them to behave as similarly as possible to avoid confusion. If it's felt useful for srcset to have maximum viewport dimensions (w/h) -- and they do seem reasonably useful -- then these should be added to image-set as well; otherwise they should be removed from srcset. Summary of recommendations. --------------------------- 1. srcset and image-set need a mechanism for dealing with flexibly-sized images, preferably by specifying the intrinsic sizes of the available images. 2. srcset's dppx restrictions should copy image-set's smart and intuitive behavior, rather than prescribing a less useful algorithm. 3. image-set should accept equivalent syntax to srcset (i.e. support viewport w/h restrictions, unless they get dropped from srcset). Of these 1 is the most important for these APIs to be a complete solution for responsive images, but 2 is also important as it makes using the APIs together significantly less confusing. Thanks for reading this far, and sorry I didn't get a chance to provide this feedback earlier, John
Received on Tuesday, 21 August 2012 19:59:52 UTC