Re: CfC: to publish "The srcset attribute" specification as a First Public Working Draft (FPWD)

Thanks Yoav, I guess I can stop rushing this proof-of-concept webkit branch
now :) I'm quite slow at C++.

I have verified that Chrome performs a layout pass immediately after CSS
parsing finishes, *prior to loading or executing any scripts*. This appears
to be where background-image usage analysis and prefetch begins, and seems
to be an ideal implementation point for a non-optimized pre-fetch algorithm.

*Width/height in markup*

The W3C and others have been training authors to specify image width/height
values for over a decade:
http://www.w3.org/wiki/Images_in_HTML#Faster_image_display_by_defining_the_dimensions_using_width_and_height

I don't think this is much of a regression in terms of maintenance or
dependencies; it's been well communicated to authors over the years, and is
trivial to implement at the CMS/WYSIYG editor level.

It should also speed up display time, as layout can occur before images are
loaded, which is important on slow connections.

> Also, I'm not sure how your proposal can enable the browser to
distinguish between art-direction and resolution switching (which is an
important part of the `srcset`/`<picture>` proposal combination) nor how
should the UA calculate DPI based resolution switching in your proposal.

*Resolution switching summary:*

1. UA has density multiplier (1.5, 1.6, 2, etc). If running in
low-bandwidth mode, it may use "1" instead.
2. UA divides all width/height values  on each `source` element by the
adjusted density multipler from #1.
3. UA selects first matching, supported image based on size constraints.

*Resolution switching vs art-direction:*

As I understand, the use case for providing two images of the same size
with different content is so users with higher-density, physically smaller
displays don't need to zoom to see the 'primary content' in the image.

I don't think this level of control will be exercised frequently, but it
could be handled via Solution B (allow 'media="" attribute, but discourage
use).

<source src="/full-image.jpg" width="640" height="480"
media="(min-width:640px)" />
<source src="/zoom-image-hires.jpg" width="640" height="480" />

*MQ DRY:*

I would also like to see a global MQB (media query breakpoint) solution.
However, it would seem strange if said solution wasn't in CSS. And if it
was in CSS, it would mean deferred MQ evaluation for markup references -
including CSS includes! I really don't think it's likely that we could get
such a solution passed. A meta-tag based approach could meet the
performance requirements, but wouldn't meet markup/presentation separation
requirements.

A generic markup-level variables solution (like HTML entities) could meet
both requirements, but wouldn't assist with CSS rules unless it was somehow
cross-spec. I can't imagine a solution that would degrade gracefully. Have
there been any proposals?

I really don't see a successful path to DRY for <picture> unless it comes
included in the <picture> spec.

The benefits for other elements such as *link*, *audio*, and *video* seem
much smaller than for *picture*, both because they will be less common, and
because their queries are simpler (video typically only cares about
viewport, link typically only cares about print/screen).

Best regards,
Nathanael


On Wed, Feb 6, 2013 at 1:29 PM, Yoav Weiss <yoav@yoav.ws> wrote:

> Nathanael,
>
> I now realize that I've read your proposal in the wrong light, at least
> partially because you stated it is based on "element size" (which I
> understood as the display dimensions of the element displaying the image).
> Having read it again, you are correct that your proposal does not require
> the browser to wait for Layout. In fact, since the CSS part of your
> proposal can be inlined, at least in theory, it can result in *zero*
> performance regression, which is a good thing.
>
> However, I'm not sure I understand the value of adding the various image
> dimensions directly into the markup. This creates a dependency between
> markup and external resources, which may burden maintenance. I do not see
> the added value in them.
> Also, I'm not sure how your proposal can enable the browser to distinguish
> between art-direction and resolution switching (which is an important part
> of the `srcset`/`<picture>` proposal combination) nor how should the UA
> calculate DPI based resolution switching in your proposal.
>
> The main advantage I do see in your proposal is DRYing up Media Queries
> from the markup. If I understand your proposal correctly, that would
> require a nested set of rules for pages that have multiple `<picture>`
> elements with various max-width per MQ breakpoint. I'm afraid that you may
> be trading HTML complexity with CSS one.
>
> In my personal opinion (which not necessarily represents the RICG) DRYing
> out Media Queries is an important effort.
> This effort is *completely orthogonal* to the `<picture>` element (but
> from which the `<picture>` element can benefit).
> Currently MQ breakpoints are defined:
> * Inside various CSS files
> * As `media` attributes for `<link>` elements
> * As `media` attributes for `<source>` elements in `<video>`/`<audio>` tags
> * As `media` attributes for `<source>` elements in `<picture>` tags
> * Possibly other elements in the future
>
> IMO A global attempt to DRY out these various MQ definitions is much
> needed. I'd love to hear future proposals from you and others on this
> issue.
> OTOH, this issue is not related to either `<picture>` or `srcset` moving
> forward to FPWD.
>
> Yoav
>
>
>
>
>
> On Wed, Feb 6, 2013 at 10:33 AM, Nathanael D. Jones <
> nathanael.jones@gmail.com> wrote:
>
>> @Yoav
>>
>> Prefetching does *not* need to be delayed until layout is calculated. I'm
>> suggesting that prefetching be performed the same way it is currently done
>> for CSS background-images. Once the markup and the css have been
>> cross-referenced, the download begins. This can occur as early as the first
>> CSS signal tag (if spec'd), the first non-media query selector after a
>> matching rule, or as late as the end of the last CSS request. From what I
>> understand about CSS parser implementations, an incremental data structure
>> is available during parsing, which should offer enough data to make an
>> informed pre-fetch decision.
>>
>> If javascript later creates a new element or changes key parameters (such
>> as min-width/min-height,max-width/max-height), then a final fetch is
>> required. This is the status quo for background-image and img prefetching,
>> so why the fixation on waiting until render for *pre*fetching?
>>
>> Pre-fetching, by nature, is optimistic. We don't wait until render to
>> pre-fetch CSS background-images, despite them being more likely to be
>> affected; why assume a worst-case browser implementation for `<picture>`?
>>
>> @Mat
>>
>> I communicate with customers implementing responsive image resizing (and
>> art direction based on face detection) on a daily basis; it's my job, and
>> RESTful image processing is my company's focus. We already have an
>> implementation based on the current <picture> spec, and frankly, the harder
>> `<picture>` is to use directly, the more people will need our help.... but
>> that's not what's best for the web.
>>
>> 1. I'm not suggesting *any* changes to CSS Media Queries whatsoever
>> 2. I'm not suggesting we consult the size of the container, rather the
>> CSS size *constraints* (min-*, max-*) of the picture element itself. Unless
>> percentage units are used, informed *pre*fetching can occur before layout.
>> 3. The naming of height/width attributes is unimportant. I would not
>> consider it repurposing, as they would behave as if they were applied to a
>> 'img' element, but if it harms backwards compatibility somehow, let's look
>> at alternate options, such as w/h and size.
>> 4. Thus my e-mail to you on the 4th, in hopes you would provide guidance
>> on the appropriate forum for this discussion.
>>
>> Best regards,
>> Nathanael Jones
>>
>>
>> On Tue, Feb 5, 2013 at 11:53 PM, Yoav Weiss <yoav@yoav.ws> wrote:
>>
>>> Nathanael,
>>>
>>> I'd like to add that you are highly underestimating the amount of delay
>>> postponing fetching of images until after the page has layout will
>>> introduce.
>>>
>>> The average delay of a 4G network is around 150 ms. For a 3G network it
>>> is around 400 ms [1]
>>>
>>> That means that in the best case scenario of a page with a single small
>>> CSS (that fits into the initial congestion window), and no External
>>> JavaScript in the head, we are looking at at least 400 ms delay in the
>>> page's overall page load for 3G users. That is a lot!
>>>
>>> Let's look at a real life loading scenario of a Web page containing
>>> several CSS files from different hosts (external font library is a common
>>> case), as well as a few external JavaScript files from various hosts (CDN
>>> based JQuery & local scripts)
>>>
>>> In this case you are looking at at least 2-3 delays more (establishing
>>> connections, requesting & fetching resources, which at least some are
>>> bigger than the 15KB of the initial congestion window), even with SPDY.
>>> Add to that CSS & JS parsing & rendering/execution, creation of DOM,
>>> RenderTree and Layout (none of which is negligable on mobile devices), and
>>> you have added 2-3 seconds to an average page.
>>>
>>> I'm afraid that the performance regression that will result from
>>> waiting for layout before requesting images is just too much for it to be
>>> a viable option.
>>>
>>> Yoav
>>>
>>> [1]
>>> http://www.igvita.com/2012/07/19/latency-the-new-web-performance-bottleneck/
>>>
>>>
>>> On Tue, Feb 5, 2013 at 11:27 PM, Mat Marquis <mat@matmarquis.com> wrote:
>>>
>>>>
>>>> On Feb 5, at 3:36 PM, Nathanael D. Jones wrote:
>>>>
>>>> I think many people were expecting a solution based on element size.
>>>>
>>>>
>>>> I would be interested in seeing data to support this assertion.
>>>>
>>>> While that may not be a use-case you've aimed to solve, *I think it
>>>> should be the *primary* use-case*, as it provides a *simple solution
>>>> to nearly every other use-case documented*.
>>>>
>>>> *Here's my full proposal:*
>>>>
>>>> https://gist.github.com/nathanaeljones/4706093
>>>>
>>>>
>>>>
>>>> Where your goal is to selectively load images based on the size of the
>>>> containing element and you’re comfortable delaying the requests for those
>>>> assets until after the layout has been painted, the functionality you’re
>>>> looking for could be easily accomplished using JavaScript and data
>>>> attributes.
>>>>
>>>> In terms of the above as a native solution: your proposal involves
>>>> heavily modifying the syntax of media queries (with syntactical overlap
>>>> between your proposal and the existing method of specifying media types),
>>>> repurposing `width` and `height` attributes, and delaying image requests to
>>>> well beyond the initial parsing of the markup (waiting until CSS and JS is
>>>> requested, transferred, and rendered to begin requesting image sources). As
>>>> a first step, I might recommend reaching out to browser representatives to
>>>> determine the implementation viability of your proposal—there are several
>>>> members of the RICG that would likely be willing to offer you feedback.
>>>>
>>>> I understand that this is a subject to which a great deal of people
>>>> have given thought. I’m happy to continue this discussion on the RICG
>>>> mailing list, but I’m sure you can understand where we won’t be withdrawing
>>>> the `picture` proposal based on proposals that haven’t yet been thoroughly
>>>> vetted or discussed at any length. The HTML WG Administrative list doesn’t
>>>> seem like the appropriate venue for those discussions.
>>>>
>>>> Thanks,
>>>> Mat Marquis
>>>>
>>>>
>>>> For the record:
>>>>
>>>> I do not believe the advantages of slightly-earlier prefetching
>>>> outweigh the benefits of a CSS-based approach. There are many possible
>>>> optimizations available to ensure the delay can be reduced to ~40ms for a
>>>> cache miss (Probably ~15ms with SPDY), and it is simply not worth the
>>>> markup complexity required.
>>>>
>>>> Best regards,
>>>> Nathanael Jones
>>>>
>>>>
>>>>  [ snip ]
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>