Re: add "networkDuration" to Resource Timing

On Mon, Nov 24, 2014 at 4:34 PM, Steve Souders <steve@souders.org> wrote:

> LONG: A few weeks ago I discovered that "duration" includes blocking time,
> so "duration" is greater than the actual network time needed to download
> the resource. Since then I've been at Velocity and WebPerfDays where many
> people have shown their Resource Timing code. Everyone I spoke to (~5
> different teams) assumed that "duration" was just the network time. When I
> explain that it also includes blocking they were surprised, admitted they
> hadn't known that, and agreed it is NOT the metric they were trying to
> capture.


Steve, can you elaborate on the use case a bit more? Who's measuring what
here, and for what purpose? Are we benchmarking CDN performance?

In terms of getting access to the data, we have the following cases:
1) same origin resources: full access to timing data.
2) different origin:
  a) if you control it, add TAO header for full access to timing data.
  b) if you don't control it, you only have "duration"

For (1) and (2a), I can see why you may want or need to get low-level
"network duration" data: you want to track your provider's DNS performance,
latency to your CDN, TTFB, total response time, and so on. You care about
this because this is something *you can affect*. However, for (2b)... this
same data falls into interesting but not actionable bucket? Further, it
seems like if you are actually interested in benchmarking your CDN, then
you really should be looking deeper than just total time: you want to
decompose DNS, TCP, TLS, HTTP req>resp cycles. At which point.. you need
the full timing object anyway.

I propose we add a new property to Resource Timing that reflects the time
> to actually load the resource excluding blocking time. I'm flexible about
> the name but for purposes of this discussion let's call it
> "networkDuration". The important piece of this proposal is that
> "networkDuration" should be available for all resources, similar to
> "duration". In other words, it should be available for same origin as well
> as cross origin resources as part of the PerformanceEntry
> <http://www.w3.org/TR/performance-timeline/#performanceentry> interface.
>

Note that "blocking time" is a thing of the past for SPDY and HTTP/2, as
this demo demonstrates really well: http://www.httpvshttps.com/

I'm skeptical of above definition: if you want "network duration", you
should also exclude cache time; it's a computed metric that you can access
today with TAO and a redundant one with http/2; if you really care about
"network duration" you should probably decompose it further, but at that
point it becomes a conversation about removing the TAO restriction.

ig

P.S. "networkDuration = dns + tcp + waiting + content" ... don't forget the
https handshake!

On Wed, Nov 26, 2014 at 9:01 AM, Patrick Meenan <pmeenan@webpagetest.org>
wrote:

> Would be great to see it either as a high-level duration or as an
> unblocking of the redirectStart time for cross-origin (though it may still
> not be clear to people that that is the time they really care about).
>
> I expect the current logic was the easiest and didn't require any privacy
> reviews because it's quite literally the exact same detail that you get if
> you do it manually in javascript by creating an element and listening to
> the onload.  Even if the more-granular detail doesn't really expose
> anything you couldn't figure out before it does provide additional detail
> that wouldn't otherwise be measurable and is probably going to require
> reviews by privacy and security teams.
>
> On Wed, Nov 26, 2014 at 9:36 AM, Peter Lepeska <bizzbyster@gmail.com>
> wrote:
>
>> +1
>>
>> On Tue, Nov 25, 2014 at 12:31 PM, Nic Jansma <nic@nicj.net> wrote:
>>
>>>  Good point!  Hadn't considered that, so yes I would agree it's a very
>>> valuable addition to consider.
>>>
>>> As far as what interface to put it on, I'm not sure networkDuration
>>> would make sense for UserTiming, for example.  While it could sit on
>>> PerformanceEntry and just be "0" for interfaces that aren't applicable, we
>>> could also create a PerformanceNetworkEntry interface (with
>>> networkDuration) that PerformanceResourceTiming inherits from, while
>>> PerformanceUserTiming only inherits from PerformanceEntry.
>>>
>>> That's all minor details though.  Really depends on the browser privacy
>>> teams OK'ing the addition.
>>>
>>> - Nichttp://nicj.net/
>>> @NicJ
>>>
>>> On 11/25/2014 12:16 PM, Steve Souders wrote:
>>>
>>> Nic -
>>>
>>> You can *not* calculate networkDuration from other attributes for
>>> *cross-origin* resources. That's why I'm suggesting adding this to
>>> PerformanceEntry (rather than PerformanceResourceTiming).
>>>
>>> And as mentioned, about 50% of resources are cross-origin so it's
>>> important to provide a means for *accurate* download time measurements.
>>>
>>> -Steve
>>>
>>>
>>> On 11/25/14, 8:02 AM, Nic Jansma wrote:
>>>
>>> Steve,
>>>
>>> The only downside I see is that we're adding a new attribute that can be
>>> entirely calculated via other attributes.
>>>
>>> One alternate (or additional thing) would be to highlight this point in
>>> the description for "duration" in the spec.
>>>
>>> - Nichttp://nicj.net/
>>> @NicJ
>>>
>>> On 11/25/2014 3:04 AM, Yoav Weiss wrote:
>>>
>>>
>>> On Tue, Nov 25, 2014 at 1:34 AM, Steve Souders <steve@souders.org>
>>> wrote:
>>>
>>>>  SHORT: I propose we add the "networkDuration" property to
>>>> PerformanceEntry
>>>> <http://www.w3.org/TR/performance-timeline/#performanceentry> objects.
>>>>
>>>> LONG: A few weeks ago I discovered that "duration" includes blocking
>>>> time, so "duration" is greater than the actual network time needed to
>>>> download the resource. Since then I've been at Velocity and WebPerfDays
>>>> where many people have shown their Resource Timing code. Everyone I spoke
>>>> to (~5 different teams) assumed that "duration" was just the network time.
>>>> When I explain that it also includes blocking they were surprised, admitted
>>>> they hadn't known that, and agreed it is NOT the metric they were trying to
>>>> capture.
>>>>
>>>> I propose we add a new property to Resource Timing that reflects the
>>>> time to actually load the resource excluding blocking time. I'm flexible
>>>> about the name but for purposes of this discussion let's call it
>>>> "networkDuration". The important piece of this proposal is that
>>>> "networkDuration" should be available for all resources, similar to
>>>> "duration". In other words, it should be available for same origin as well
>>>> as cross origin resources as part of the PerformanceEntry
>>>> <http://www.w3.org/TR/performance-timeline/#performanceentry>
>>>> interface.
>>>>
>>>> Same origin resources can calculate "networkDuration" as follows
>>>> (assume "r" is a PerformanceResourceTiming
>>>> <http://?ui=2&ik=b493d86064&view=att&th=149e4608a5dad0d6&attid=0.1.1&disp=emb&zw&atsh=0>
>>>> object):
>>>>
>>>>     dns = r.domainLookupEnd - r.domainLookupStart;
>>>>     tcp = r.connectEnd - r.connectStart;        // includes ssl
>>>> negotiation
>>>>     waiting = r.responseStart - r.requestStart; // aka "TTFB"
>>>>     content = r.responseEnd - r.responseStart;
>>>>     networkDuration = dns + tcp + waiting + content;
>>>>
>>>> I've discussed this with a few people and the only concern I've heard
>>>> is with regard to privacy along the lines of "if we exclude blocking we've
>>>> added the ability to distinguish cache reads from network fetches". This
>>>> isn't an issue for two reasons:
>>>>
>>>>    1. Even with the exclusion of blocking time, it's still possible
>>>>    for "networkDuration" to have a non-zero value for resources read from
>>>>    cache due to disk access time, etc. Therefore, excluding blocking time does
>>>>    not necessarily provide a clear means of determining resources read from
>>>>    cache.
>>>>    2. This concern assumes that adding "networkDuration" lessens
>>>>    privacy because removing blocking time provides additional information that
>>>>    is not available today. However, it's possible to exclude blocking time
>>>>    today by loading a cross-origin resource after window.onload, when there is
>>>>    no blocking contention.
>>>>
>>>> Therefore, individuals who have JavaScript access to a page and can
>>>> measure duration also have enough access to load resources after
>>>> window.onload and can thus determine the duration excluding blocking time.
>>>> Adding "networkDuration" does not give these individuals additional
>>>> information beyond what is measurable today.
>>>>
>>>> What "networkDuration" provides is additional information for the
>>>> normal case of resources that are loaded as part of the main page when
>>>> blocking contention may occur. This will give current web developers the
>>>> metric they want for cross-origin resources, and will provide it more
>>>> simply for same origin resources.
>>>>
>>>
>>>  Assuming that the privacy concerns are in fact non-existent, a big +1.
>>>
>>>
>>>
>>>
>>>
>>
>