RE: CfC: to publish "The picture element" specification as a First Public Working Draft (FPWD)

Dear Mat,

Thank you for following up.  It does seem that we are in basic agreement on the approach.

Are the proponents of the original srcset proposal happy with it becoming a resolution hint and for the syntax to be optimized as the proposal develops for this purpose and for the use of srcset for media query based image choice to be excluded and handled only by the <source media> element attribute?

Perhaps it would be best to edit the srcset proposal to be more consistent with the plan.  For example, remove the viewport size media query parameter from the syntax.  There were quite a few good ideas and discussion on the whatwg list for using srcset for this purpose and surely the syntax could be usefully extended.

Some concerns raised with using the srcset as a resolution hint were: the image box size might not be know at the time the image could be loaded; the image resolution affects the layout.   Some consideration of these concerns would be useful.

Section 8 'Adaptive images' of the srcset proposal appears inconsistent with the intent of the proposal.  There are examples using the srcset as a viewport media query to select between images.  Perhaps just remove this section.

cheers
Fred

From: mat@matmarquis.com
Date: Sat, 2 Feb 2013 19:38:55 -0500
CC: public-html-admin@w3.org; public-respimg@w3.org; public-pua@w3.org
To: fredandw@live.com
Subject: Re: CfC: to publish "The picture element" specification as a First  Public Working Draft (FPWD)


On Feb 2, at 6:39 PM, Fred Andrews wrote:I object to this being publish as a FPWD.

The design of the picture element and srcset both fail to address the most fundamental use case which is giving the UA a choice in the selection of the image resolution to use.  This choice is needed to meet two critical use cases: 1. allowing lower resolution images to be chosen when the UA considers them adequate and to reduce load time and bandwidth usage; 2. allowing higher resolution images to be chosen when the UA considers them needed to give sharp presentation on high DPI screens or when otherwise required based on image box size or transforms or when content is magnified.

Both proposals fail to meet these fundamental needs.  Both proposals dictate the resource choice based on media query parameters which have a weak relationship to the actual needed resource.  Further, dictating the choice gives the UA no choice in the decision and no room to innovate.  For example a UA can not confidently decide to use lower resolution images to save bandwidth and reduce page load time or to offer the user a 'click to sharpen' action on images which downloads a higher resolution image.  For example, a UA can not choose to downscale a locally cached high resolution image rather than downloading a lower resolution image.

`srcset` provides a microsyntax that does exactly that—the resolution sources/thresholds can be overridden by the UA based (theoretically) on client preferences or bandwidth available (see the note under step 17 of the processing algorithm on http://www.w3.org/html/wg/drafts/srcset/w3c-srcset/#processing-the-image-candidates, as well as the last paragraph under http://picture.responsiveimages.org/#algorithm-for-deriving-the-source-image ).
The primary focus of `srcset`—both on `img` and on a `source` element—is a microsyntax that allows authors to specify “suggested” resolution-based sources. 
The `picture` element, which does make use of `media` attributes, is intended more for serving appropriate image source *dimensions* as well as sources with differing crop and zoom points for the sake of smaller displays: the “art direction” use case. We wouldn’t want the UA to suggest an image source with smaller dimensions, as this may skew the page’s layout, provide an image inappropriately sized for the containing element(s), etc.
Other proposals have been discussed in length on the whatwg mailing list and Ian Hickson has failed to integrate the suggestions or meet these fundamental use cases.  Other people have pointed out the unnecessary complexity for web content developers in trying to used these proposals which I believe is caused by conflating the image resolution choice with media query art-direction choices etc and the inherent weak relationship between the media query parameters and the required image resolution.

`srcset` provides the exact functionality you describe with regards to resolution requirements (beyond screen density, including bandwidth, user preference, etc.). The `picture` element serves the “art direction” use case, using a familiar syntax that uses media queries to match up with the layout as-specced—relative units, min/max-width media queries, and so on. The two concerns aren’t conflated.
It may help to think of the combined syntax this way: first, make use of MQ syntax to select the appropriate `source` element. Then, choose from the *suggested* sources contained within that `source` element’s `srcset` attribute. The author can *suggest* what sources might be used given the client’s screen density, but the UA, a user preference, or an “available bandwidth” threshold will have the opportunity to intervene and request the lower resolution images, as that’s where the real bandwidth damage would be done.
A redesign needs to start by separating the image resolution choice from and 'media query' parameters.  This resolution choice is fundamentally orthogonal to the 'art direction' etc use cases.  This choice can not be based on a media query of the screen DPI or viewport size as this has a weak relationship to the needed image resolution, and the choice can not be dictated in the standard as this gives the UA no choice.  If the UA is to have choice then the set of images must be simple scaled versions of the same image. There are lots of other more minor issues to be worked through by they should be easier to reach consensus on.  A media query based selection proposal could work orthogonally for the art-direction use cases etc.

What you’ve described here is the very reasoning behind the two disparate syntaxes. I actually think we may have some crossed wires—we’re in agreement on all of the points you make above, and I feel strongly that the two syntaxes working in concert address your concerns.
This write-up is a little dated at this point, but does explain how we (the Responsive Images Community Group) arrived at the combined syntax: http://www.w3.org/community/respimg/2012/06/18/florians-compromise/
Yoav Weiss, also of the RICG, expanded on the syntax would allow the browser to perform heuristic optimizations in this post: http://mobile.smashingmagazine.com/2013/01/09/bandwidth-media-queries-we-dont-need-em/
Your paragraph above could very well be an excerpt from either of those posts; I’m confident we’re on the same page.
Solutions to these use cases are needed urgently and this process has dragged on far too long.   A review of the process may be in order.

In the vacuum created by the failure to meet these use cases other proposals to address the matter on the server-side are emerging, such as the Client-Hints proposal.  Addressing the issue on the server-size requires the UA to unnecessarily share a lot of state and this is a privacy concern.   As Chair the PUA CG this 'required' leaking of UA state is a great concern and I would like the HTML support for image adaptation to be solved on the client side so that the server-side solutions can take advantage of it and transition to using client-side image choice.

I couldn’t agree more on this point. Please don’t hesitate to follow-up—again, I think we’re in agreement, and I would hate for an objection to carry based on a miscommunication on my part.
Thanks,Mat Marquis