Re: CfC: to publish "The srcset attribute" specification as a First Public Working Draft (FPWD)

Nathanael,

I now realize that I've read your proposal in the wrong light, at least
partially because you stated it is based on "element size" (which I
understood as the display dimensions of the element displaying the image).
Having read it again, you are correct that your proposal does not require
the browser to wait for Layout. In fact, since the CSS part of your
proposal can be inlined, at least in theory, it can result in *zero*
performance regression, which is a good thing.

However, I'm not sure I understand the value of adding the various image
dimensions directly into the markup. This creates a dependency between
markup and external resources, which may burden maintenance. I do not see
the added value in them.
Also, I'm not sure how your proposal can enable the browser to distinguish
between art-direction and resolution switching (which is an important part
of the `srcset`/`<picture>` proposal combination) nor how should the UA
calculate DPI based resolution switching in your proposal.

The main advantage I do see in your proposal is DRYing up Media Queries
from the markup. If I understand your proposal correctly, that would
require a nested set of rules for pages that have multiple `<picture>`
elements with various max-width per MQ breakpoint. I'm afraid that you may
be trading HTML complexity with CSS one.

In my personal opinion (which not necessarily represents the RICG) DRYing
out Media Queries is an important effort.
This effort is *completely orthogonal* to the `<picture>` element (but from
which the `<picture>` element can benefit).
Currently MQ breakpoints are defined:
* Inside various CSS files
* As `media` attributes for `<link>` elements
* As `media` attributes for `<source>` elements in `<video>`/`<audio>` tags
* As `media` attributes for `<source>` elements in `<picture>` tags
* Possibly other elements in the future

IMO A global attempt to DRY out these various MQ definitions is much
needed. I'd love to hear future proposals from you and others on this
issue.
OTOH, this issue is not related to either `<picture>` or `srcset` moving
forward to FPWD.

Yoav





On Wed, Feb 6, 2013 at 10:33 AM, Nathanael D. Jones <
nathanael.jones@gmail.com> wrote:

> @Yoav
>
> Prefetching does *not* need to be delayed until layout is calculated. I'm
> suggesting that prefetching be performed the same way it is currently done
> for CSS background-images. Once the markup and the css have been
> cross-referenced, the download begins. This can occur as early as the first
> CSS signal tag (if spec'd), the first non-media query selector after a
> matching rule, or as late as the end of the last CSS request. From what I
> understand about CSS parser implementations, an incremental data structure
> is available during parsing, which should offer enough data to make an
> informed pre-fetch decision.
>
> If javascript later creates a new element or changes key parameters (such
> as min-width/min-height,max-width/max-height), then a final fetch is
> required. This is the status quo for background-image and img prefetching,
> so why the fixation on waiting until render for *pre*fetching?
>
> Pre-fetching, by nature, is optimistic. We don't wait until render to
> pre-fetch CSS background-images, despite them being more likely to be
> affected; why assume a worst-case browser implementation for `<picture>`?
>
> @Mat
>
> I communicate with customers implementing responsive image resizing (and
> art direction based on face detection) on a daily basis; it's my job, and
> RESTful image processing is my company's focus. We already have an
> implementation based on the current <picture> spec, and frankly, the harder
> `<picture>` is to use directly, the more people will need our help.... but
> that's not what's best for the web.
>
> 1. I'm not suggesting *any* changes to CSS Media Queries whatsoever
> 2. I'm not suggesting we consult the size of the container, rather the CSS
> size *constraints* (min-*, max-*) of the picture element itself. Unless
> percentage units are used, informed *pre*fetching can occur before layout.
> 3. The naming of height/width attributes is unimportant. I would not
> consider it repurposing, as they would behave as if they were applied to a
> 'img' element, but if it harms backwards compatibility somehow, let's look
> at alternate options, such as w/h and size.
> 4. Thus my e-mail to you on the 4th, in hopes you would provide guidance
> on the appropriate forum for this discussion.
>
> Best regards,
> Nathanael Jones
>
>
> On Tue, Feb 5, 2013 at 11:53 PM, Yoav Weiss <yoav@yoav.ws> wrote:
>
>> Nathanael,
>>
>> I'd like to add that you are highly underestimating the amount of delay
>> postponing fetching of images until after the page has layout will
>> introduce.
>>
>> The average delay of a 4G network is around 150 ms. For a 3G network it
>> is around 400 ms [1]
>>
>> That means that in the best case scenario of a page with a single small
>> CSS (that fits into the initial congestion window), and no External
>> JavaScript in the head, we are looking at at least 400 ms delay in the
>> page's overall page load for 3G users. That is a lot!
>>
>> Let's look at a real life loading scenario of a Web page containing
>> several CSS files from different hosts (external font library is a common
>> case), as well as a few external JavaScript files from various hosts (CDN
>> based JQuery & local scripts)
>>
>> In this case you are looking at at least 2-3 delays more (establishing
>> connections, requesting & fetching resources, which at least some are
>> bigger than the 15KB of the initial congestion window), even with SPDY.
>> Add to that CSS & JS parsing & rendering/execution, creation of DOM,
>> RenderTree and Layout (none of which is negligable on mobile devices), and
>> you have added 2-3 seconds to an average page.
>>
>> I'm afraid that the performance regression that will result from
>> waiting for layout before requesting images is just too much for it to be
>> a viable option.
>>
>> Yoav
>>
>> [1]
>> http://www.igvita.com/2012/07/19/latency-the-new-web-performance-bottleneck/
>>
>>
>> On Tue, Feb 5, 2013 at 11:27 PM, Mat Marquis <mat@matmarquis.com> wrote:
>>
>>>
>>> On Feb 5, at 3:36 PM, Nathanael D. Jones wrote:
>>>
>>> I think many people were expecting a solution based on element size.
>>>
>>>
>>> I would be interested in seeing data to support this assertion.
>>>
>>> While that may not be a use-case you've aimed to solve, *I think it
>>> should be the *primary* use-case*, as it provides a *simple solution to
>>> nearly every other use-case documented*.
>>>
>>> *Here's my full proposal:*
>>>
>>> https://gist.github.com/nathanaeljones/4706093
>>>
>>>
>>>
>>> Where your goal is to selectively load images based on the size of the
>>> containing element and you’re comfortable delaying the requests for those
>>> assets until after the layout has been painted, the functionality you’re
>>> looking for could be easily accomplished using JavaScript and data
>>> attributes.
>>>
>>> In terms of the above as a native solution: your proposal involves
>>> heavily modifying the syntax of media queries (with syntactical overlap
>>> between your proposal and the existing method of specifying media types),
>>> repurposing `width` and `height` attributes, and delaying image requests to
>>> well beyond the initial parsing of the markup (waiting until CSS and JS is
>>> requested, transferred, and rendered to begin requesting image sources). As
>>> a first step, I might recommend reaching out to browser representatives to
>>> determine the implementation viability of your proposal—there are several
>>> members of the RICG that would likely be willing to offer you feedback.
>>>
>>> I understand that this is a subject to which a great deal of people have
>>> given thought. I’m happy to continue this discussion on the RICG mailing
>>> list, but I’m sure you can understand where we won’t be withdrawing the
>>> `picture` proposal based on proposals that haven’t yet been thoroughly
>>> vetted or discussed at any length. The HTML WG Administrative list doesn’t
>>> seem like the appropriate venue for those discussions.
>>>
>>> Thanks,
>>> Mat Marquis
>>>
>>>
>>> For the record:
>>>
>>> I do not believe the advantages of slightly-earlier prefetching outweigh
>>> the benefits of a CSS-based approach. There are many possible optimizations
>>> available to ensure the delay can be reduced to ~40ms for a cache miss
>>> (Probably ~15ms with SPDY), and it is simply not worth the markup
>>> complexity required.
>>>
>>> Best regards,
>>> Nathanael Jones
>>>
>>>
>>>  [ snip ]
>>>>
>>>>
>>>>
>>>
>>>
>>
>