- From: Ian Hickson <ian@hixie.ch>
- Date: Tue, 4 Sep 2012 19:47:38 +0000 (UTC)
- To: "whatwg@whatwg.org" <whatwg@whatwg.org>
- Message-ID: <Pine.LNX.4.64.1209040049490.30734@ps20323.dreamhostps.com>
On Thu, 7 Jun 2012, Mark Callow wrote:
> On 06/06/2012 21:36, Henri Sivonen wrote:
> > More to the point, the important characteristic is being able to stop
> > downloading *quarter* way through the file and get results that are as
> > good as if the full-size file had been down sampled with both
> > dimensions halved and that size had been sent as the full file. (I am
> > not aware of a bitmap format suitable for photographs that has this
> > characteristic. I am aware that JPEG 2000 does not have this
> > characteristic. I believe interlaced PNGs have that characteristic,
> > but they aren't suitable for photographs, due to the lossless
> > compression.)
>
> IIRC Kodak's PhotoCD image format had this characteristic. The first
> part is a low res. image, the second is the differences between the low
> res. image zoomed to the high res. size and the actual high res. image.
Such formats certainly would make a lot of this simpler!
On Wed, 8 Aug 2012, Markus Ernst wrote:
> > >
> > > - Use <picture> for the art direction use case.
> > > - Remove the MQ except the pixel density from @srcset, and make
> > > @srcset available in both <img> and <source>:
> > >
> > > <img src="normal.jpg" alt="" srcset="high.jpg 2x, normal.jpg 1x">
> > >
> > > <picture alt="">
> > > <source src="mobile.jpg"
> > > srcset="low.jpg 0.5x, mobile.jpg 1x, mobile-hd.jpg 2x">
> > > <source src="large.jpg" media="min-width: 600px"
> > > srcset="large.jpg 1x, large-high.jpg 2x">
> > > <img src="mobile.jpg">
> > > </picture>
> >
> > I don't understand why this is better than:
> >
> > <img src="large.jpg" alt=""
> > srcset="low.jpg 600w 0.5x, mobile.jpg 600w 1x, mobile-hd.jpg 600w 2x,
> > large-high.jpg 2x">
> >
> > It seems to me that srcset as specified does exactly the same thing,
> > but more succintly.
>
> [...]
>
> > On Thu, 24 May 2012, Florian Rivoal wrote:
> > >
> > > <picture>
> > > <source srcset="normal.jpg 1x, highres.jpg 2x">
> > > <source media="(max-width:768px)" srcset="ipad.jpg 1x, ipad3.jpg 2x">
> > > <source media="(max-width:320px)" srcset="iphone.jpg 1x, iphone4.jpg
> > > 2x">
> > > <img src="normal.jpg">
> > > </picture>
> >
> > I don't understand why this is better than:
> >
> > <img src="normal.jpg" srcset="highres.jpg 2x,
> > ipad.jpg 768w 1x, ipad3.jpg 768w 2x,
> > iphone.jpg 320w 1x, iphone4.jpg 320w 2x"
> > alt="...">
> >
> > ...which as far as I can tell does exactly the same thing.
>
> It is better because art direction and bandwidth use cases can be solved
> differently in an appropriate manner:
> - For the bandwidth use case, no MQ is needed, but only some information on
> the sources available to let the UA decide which source to load.
> - For the art direction use case OTOH, the <picture> element is more intuitive
> to handle and also easier to script, as sources can be added or removed via
> DOM.
I don't understand why it's more intuitive and easier. It seems way more
unwieldly.
Whether it's easier for script is hard for me to say, because I don't
really understand what scripts are going to be doing here. Can you
elaborate? What will scripts need to do here?
If it is harder to script, we can always provide a dedicated API.
Manipulating <picture> from script would be a huge pain -- you'd have to
be manipulating lots of elements and attributes.
> Anyway, with your proposal, would this be valid, to address the
> bandwidth-only use case?:
>
> <img src="normal.jpg" alt="" srcset="high.jpg 2x, normal.jpg 1x">
Yes (assuming it's a presentational image). You can make it even shorter:
<img src="normal.jpg" alt="" srcset="high.jpg 2x">
On Thu, 9 Aug 2012, Andy Davies wrote:
>
> I know it's there for graceful degradation reasons but doesn't setting
> the 1x image via src and the others via srcset just add to developers
> confusion?
Doesn't seem that confusing to me, but maybe I've more confidence in
authors than you? :-)
> Would also like to see if there's a way of using srcset to hint to the
> UA that it can skip the image under low throughput conditions e.g. GPRS.
> Same would apply to image-set in CSS
The browser is never required to show the image, so that's already
possible for all images. However, for a discussion of the difficulties of
bandwidth-driven negotiations, see:
http://lists.w3.org/Archives/Public/public-whatwg-archive/2012May/0247.html
On Wed, 8 Aug 2012, Florian Rivoal wrote:
>
> 1) Your syntax (almost, see point 2) replicates the behavior of
> max-width and and max-height Media Queries, but doesn't give access to
> other existing and future media queries, some of which may actually be
> useful. For example:
>
> a) using the 'monocrhome' MQ to serve gray scale images to
> black-and-white printers or e-ink displays. Displaying a color image on
> a monochrome display does not always work well, as two different colors
> of similar luminosity would be impossible to distinguish in a monochrome
> environment. I expect this need to grow together with the increasing
> popularity of HTML based ebooks.
Is this a real use case or a theoretical one? Until we didn't support it,
nobody once mentioned that it was a use case they cared about -- they only
mentioned dimensions as being the issue.
> b) Microsoft is proposing a MQ which lets you detect that the UA has been put
> in hight contrast mode (for accessibility reasons), and that your content
> should follow along.
> http://msdn.microsoft.com/en-us/library/windows/apps/hh465764.aspx
> Being able to switch images based on that could be nice.
If this becomes something that there is authoring demand for, it would be
easy for us to add support for it. (Are there many sites that provide
high-contrast alternatives today?)
> c) using the 'orientation' MQ to pick a different cropping of an image,
> in a way that is similar to the width-only decision in the Obama
> example.
When would you change the image based on orientation rather than
dimensions? If you have any real-world examples of this it would be quite
helpful.
> 2) in the srcset-only notation, w and h are hints for the browser, since
> picking the right image is something for which browsers are given the
> option of doing it differently. For the pixel density of the image,
> leaving it up to the browser is a good thing, as there are a large
> number of factors that can potentially come into play. For w and h
> though, this is pretty much entirely up to the author's discretion, so
> allowing the browser to have different rules for picking the image
> doesn't buy us anything. If id didn't cost anything either, that'd be
> ok, but it does.
I don't understand. The spec seems to have this distinction for srcset, it
doesn't seem to me to be different than media queries in an important way.
> 3) you syntax is terser, which is in generally a good thing, but I think
> it crosses the limit, as a large number of people have expressed
> confusion as to w and h were min or max, for example. The extra
> verbosity of my syntax gets you an extra bit of clarity, admittedly at
> the cost of having multiple elements.
I agree that there's a small learning curve, but it seems pretty easy to
understand. Do we really want to trade the small learning curve for a
perpetuity of verbosity?
> I have always been comfortable with the 'x' part of srcset, but the w
> and h part felt somewhat wrong to me. What you'd really want to consider
> when deciding which image to pick isn't the size of the viewport itself,
> but the size available for the image once the rest of the layout is
> taken into account.
Yeah. That's how I originally designed srcset="", actually, but it was
pointed out to me that that's impossible to implement because at the time
the browsers need to pick an image, they haven't yet gotten the style
sheet so they don't know what the layout will be.
(Note that the media-query-based solutions have the same problem.)
> Because of that, I was interested in proposals involving an (optional)
> defer attribute, which let you wait until layout can inform the
> decision. Then, given accurate meta-data on the images's sizes and
> density, it would be possible to pick the optimum one. But of course,
> this means waiting, so this isn't generally acceptable and must be opt
> it. And I can't put my finger on exactly how that would work either.
If this is something people want to do, it's easy enough to script. If it
turns out that people are often trading the performance for this effect,
we can definitely look into adding it as a supported feature, but I think
(or rather, browser vendors seem to think) it's more likely that the
performance matters more than the convenience.
> Having established that perfection cannot be reached here, I could now
> consider whether srcset was good enough for most cases, and I think that
> if we don't care about things like 'monochrome', 'high-contrast', or
> 'orientation', it is.
>
> To me, the only remaining question is whether we care about these. If we
> do, rather than reinventing a new syntax for every media query someone
> wants to use in a srcset, I think we're better off letting people use
> media queries directly, in which case then w and h are a redundant
> terser syntax for max-width and max-height. If we don't, srcset is
> probably as good as it gets.
I more or less agree, except that I think the verbosity of media queries
repeated a series of times with only a URL for each case is a _lot_ of
verbosity, and probably does deserve a terse syntax even if we add other
features. It's not like style sheets, where you have one media query per
link and so each one is present once per page, or @media rules where you
typically have one per _site_ for each media query -- images occur a _lot_
on a site and so you can end up with many media queries repeated many
times per page.
(Incidentally, lest anyone think I have "not invented here" syndrome for
media queries or something, note that I was closely involved in their
development and came up with several bits of their esoteric syntax.)
On Thu, 9 Aug 2012, Kornel Lesi¨˝ski wrote:
>
> Fractional ratios have proven to be unnecessary: on desktops 1x CSS
> pixel changed from 72dpi (CRT) to 130dpi on notebook screens, but we
> haven't got fractional scaling ratios along the way.
That's not entirely accurate. It's true only for users not using the
"zoom" feature in browsers.
On Mon, 13 Aug 2012, Henri Sivonen wrote:
>
> If it indeed is the case that there are really only two realistic
> bitmaps samplings for catering to differences in weeding device pixel
> density (ignoring art direction), it would make sense to have simply
> <img src="1xsampling.jpg" hisrc="2xsampling.jpg" alt="Text alternative">
> instead of an in-attribute microsyntax for the non-art-directed case.
If that was the only use case, indeed. But I'm not sure the first
assumption is valid, and the second one is clearly not.
> >> I'm not convinced that it's a good idea to solve these two axes in
> >> the same syntax or solution. It seems to me that srcset="" is bad for
> >> the art-direction case and <picture> is bad for the pixel density
> >> case.
> >
> > I don't really understand why
>
> They are conceptually very different: One is mere mipmapping and can be
> automatically generated. The other involves designer judgment and is
> conceptually similar to CSS design where authors use MQ. Also, having w
> and h refer to the browsing environment and x to the image in the same
> microsyntax continues to be highly confusing.
The "w", "h", and "x" all refer to both the environment and the image.
They describe what the environment's characteristics need to be for the
image to be useful.
> Ignoring implementation issues for a moment, I think it would be
> conceptually easier it to disentangle these axes like this:
>
> Non-art directed:
> <img src="1xsampling.jpg" hisrc="2xsampling.jpg" alt="Text alternative">
>
> Art directed:
> <picture>
> <source src="1xsampling-cropped.jpg" hisrc="2xsampling-cropped.jpg"
> media="max-width: 480px">
> <img src="1xsampling.jpg" hisrc="2xsampling.jpg" alt="Text alternative">
> </picture>
Fundamentally, a multiple-element solution here is simply a non-starter,
IMHO. The pros of the multielement solution with verbose media queries are
about the same in magnitude as the pros of the one-attribute solution with
terse syntax, but the cons of the terse syntax are small whereas the cons
of the multiple-element syntax are immense. For the multi-element solution
to be a net positive over the one-attribute solution, the magnitude of its
"pros" would have to be enormous.
On Tue, 21 Aug 2012, Mathew Marquis wrote:
> On Aug 7, 2012, at 3:09 PM, Ian Hickson wrote:
> > On Tue, 15 May 2012, Matthew Wilcox wrote:
> >>
> >> I do not see much potential for srcset. The result of asking the
> >> author community was overwhelmingly negative, indirection or no
> >> indirection.
> >
> > I'm happy to consider specific feedback, but at the time in this
> > thread where this e-mail was sent, we had yet to receive any negative
> > commentary at all on the mailing list, so this seems a bit vague. :-)
>
> A great deal of author feedback was posted to
> http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/, the
> link to which was posted to the mailing list several times. You’ll
> notice that the comments are nearly unanimous.
I did go through those at the time.
> If you’d prefer I instruct the developer community to comment on this
> thread instead, I’m happy to do so. If author sentiment was somehow
> ignored the first time, though — if it either went wholly unnoticed by
> the decision-makers, or is now being disregarded due to where those
> sentiments were posted — I don’t see where it will change much apart
> from several hundred more emails to prove a fairly self-evident point.
The quantity of feedback is irrelevant. I only consider the weight of the
arguments.
However, if people have information that has not received a reply, then
yes, please e-mail it to the list. While I do read all the input I am
aware of (including the many posts to the CG), I only guarantee a response
to e-mails sent to this list.
On Wed, 22 Aug 2012, John Mellor wrote:
>
> The primary problem with using w/h tokens for this, and the reason the
> above srcset is so redundant, is that in the current srcset spec the x
> tokens are totally independent from the w/h tokens. So if you have an
> image that's 640 image pixels wide, it is suitable for display 640 CSS
> px wide on a low dpi device, but only 480 CSS px wide on a medium dpi
> device, and only 320 CSS px wide on a high dpi device, etc.
It is true that if you do in fact want to use the same image for different
width/density combinations, that you will have to repeat the filename. But
it seems highly unlikely that you'd want to use the same image on a 250
CSS pixel wide device at 4x as on a 1000 CSS pixel wide screen at 1x. On
the small device, the image would be tiny!
> Finally you'll notice that I put "320.jpg 320w 1.25x" instead of
> "320.jpg 320w 1x". That's because (as pointed out in 2.) srcset is
> currently specified such that these are strict maximums, yet in practice
> it's better to upscale an image a little bit (e.g. 1.25x) than to
> download an image that's twice as big (potentially 4x the file size).
> Cobbling together w/h and x tokens like this requires the author to
> manually make value judgements like this and incorporate them into their
> calculations, when it would be better to let the browser decide.
Actually doing this with srcset="" would not get the effect you want, as
the "x" value affects the image's intrinsic dimensions. You should
describe the image's ideal environment for best results.
However, nothing stops the user agent from using the 1x image when it's
really at 1.25x (e.g. due to zoom). It's up to the UA to make that
judgement call.
On Wed, 22 Aug 2012, John Mellor wrote:
>
> So for example "1280.jpg 4x" means that this image is 4 times larger
> than the given intrinsic width of 320px. So sure, it would be suitable
> for display on a hypothetical 4x display at 320px width; but the browser
> also knows that it would be suitable for display on a 2x display at
> 640px width, a 1.5x display at 853px width, and a 1x display at 1280px
> width.
This isn't accurate. A trivial example of it not being accurate is a 1000
device pixel image that consists of a horiontal double-headed arrow
labeled "Five Hundred CSS Pixels". That image is _only_ applicable in a
500 CSS pixel double-density environment. If you sed it in a 250 CSS pixel
quad-density environment, it would be wrong.
> As I said, the fact that UAs have freedom doesn't change the semantics
> in the eyes of the author. The description and algorithm in the spec
> heavily imply that a typical UA algorithm will be to choose the images
> with the smallest pixel size descriptors >= the pixel size of the
> display. It would be much clearer if this implication was not made, i.e.
> remove step 21 completely, change step 24 to read "Remove all the
> entries in candidates except the one whose associated pixel density is
> the best match for a user-agent-defined value giving the nominal pixel
> density of the display.", and change the phrase "maximum pixel density"
> to just "pixel density" in [the spec].
The idea is to provide a "default" algorithm that will usually be used,
while still allowing the kinds of lee-way you describe (note in particular
that the number is compared to "a *user-agent-defined* value giving the
nominal pixel density of the display" (emphasis added)). Most of the time,
when given a choice between upscaling a low-res image and down-scaling a
high-res one, there's only one good choice.
On Wed, 22 Aug 2012, John Mellor wrote:
>
> The w/h descriptors are sufficient *only* for art direction, where you
> want to swap between completely different images (or at least different
> crops of the same image), based on viewport size, to match other changes
> occurring in your responsive design as the viewport size changes (and
> they do a fine job at art direction).
This is unsurprising, since this is exactly the use case for which they
were designed.
> They are *not* suitable (even combined with x descriptors) as a general
> purpose mechanism for dealing with flexibly-sized images, where you need
> to match the number of device pixels taken up by the image to the number
> of image pixels in the image you load (while allowing the browser to
> adjust this based on bandwidth constraints etc), since doing that
> properly *requires* additional information, i.e. knowing the intrinsic
> size (or at least intrinsic width or height) of the image before
> choosing which to download.
This is true, because this use case was not one of the use cases we
attempted to address. This is the first I've even heard that it might be
interesting, in fact. Do you have any examples of Web pages where a
low-res desktop image makes sense as a high-res mobile image, say?
> The art direction use case can be entirely orthogonal. It should be
> handled with the w/h descriptors as currently specified. What I'm
> proposing would operate after any w/h descriptors have narrowed down the
> set of allowable images, and let the browser choose between the
> remaining images more intelligently in the case of flexible-size images,
> where currently the browser has no idea which to use.
Could you give a real example of this kind of thing? I'd love to study
what kinds of images we're talking about here. None of the examples people
posted when we were designing srcset="" were like this.
On Sat, 25 Aug 2012, Fred Andrews wrote:
>
> Agreed. The width and height of the images is what is needed. The
> browser needs to fill pixels so needs to know the sizes of the available
> images in pixels.
>
> Forget the x scale descriptions and all the other proposed extensions
> and just include a set of images with their pixel height and width.
> Forget dpi etc, its just not needed for this.
I don't understand this proposal at all. Can you show a concrete example
of how you see this working?
> Consider adding a 'selected' flag to give JS control of the selection.
> If a particular image is 'selected' then it is the default choice. If no
> image is 'selected' then the browser can use a default algorithm to make
> a choice based on the target image box size and user choices relating to
> image quality versus bandwidth tradeoffs. This allows JS to override
> the browser default algorithm. For example a web gallery can offer
> visitors a 'quality' choice and use this to flag an appropriate selected
> image.
This is definitely something we can add if it turns out to be something
people want.
> Image cropping and centering is something best left to JS algorithms.
> Fluid cropping and centering may need an algorithm that takes into
> account particular areas of interest and focus within an image, and user
> choices which would be better handled within the webpage and not at the
> browser level. A web gallery may allow visitors to choose to scale
> versus crop images and to pan cropped image.
Actually panning and so on would presumably best be done using <canvas>. I
don't think we need to be providing a declarative way to do this.
> Regarding bandwidth choices, is there are proposal that the image sizes
> in bytes could be declared in the HTML, or is there an expectation that
> the browser will query the server for the size of each image, or would
> web browser just use a heuristic to estimate the image byte size scaling
> based on the image dimensions?
For a discussion about bandwidth issues, please see:
http://lists.w3.org/Archives/Public/public-whatwg-archive/2012May/0247.html
On Tue, 21 Aug 2012, John Mellor wrote:
>
> CSS image-set and HTML img srcset are getting their first
> implementations but both APIs have serious shortcomings.
First versions of things always do. That's fine. :-)
> We should fix them before it's too late
It's rarely too late for us to add new features.
> 2. srcset not as smart/intuitive as image-set.
>
> The syntax for HTML img srcset and CSS image-set is very similar, thus
> developers will assume they behave the same. This would be great, except
> that they actually behave rather differently. For example, compare:
>
> <img srcset="low.jpg 1x, high.jpg 2x">
> selector { background: image-set("low.jpg" 1x, "high.jpg" 2x); }
>
> Surely such a simple case must behave the same? But actually, if you
> view this on a device whose dppx is 1.1, the image-set will most likely
> load low.jpg, as it is the closest match for the display, while the
> srcset will load high.jpg, because the specified algorithm treats the
> dppx values as strict maximums.
This is incorrect. The HTML spec allows the user agent to pick what it
will pretend the device density to be. Which it picks is up to the UA.
On Tue, 21 Aug 2012, Tab Atkins Jr. wrote:
>
> Currently, the CSS image-set() function (sister spec to <img srcset>) is
> specified to do fallback to the less-desirable options if the first
> choice doesn't load or can't be decoded. I specified it this way
> because it seemed reasonable, and it's how the similar image() function
> works (which is specifically designed to be used for fallback).
>
> It was later pointed out to me, though, that @srcset doesn't do any
> fallback. Once it chooses a candidate image, it sticks with it, and if
> that image is invalid, you just get a broken image.
>
> I'd like to harmonize the two, and the CSSWG seems to prefer the
> fallback behavior. Is there any interest in making @srcset do similar
> fallback? If not, I'll change image-set() to also not do fallback.
In general, I think it's fine for them to differ here. CSS is opaque in
that it does fallback silently, e.g. you can't easily tell which font it
has picked, or which media query it has picked. HTML on the other hand is
verbose in its announcement of events, and this will likely only grow
(e.g. people have already asked for progress events in <img>), so as a
rule we try to only do a single image at a time.
Having said that, technically srcset="" allows the browser to restart its
algorithm at any time, so a browser could do fallback if it wanted to.
So I would leave them as is.
--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Received on Tuesday, 4 September 2012 19:48:12 UTC