Re: [Resource Timing] Unified Timing v3

Nice work! I really like this a lot.

I see your point about marks and measures. Do we have a known case where we
expect the developer to only uses marks or measures, but nothing else? If
so, then this definitely seems worthwhile. If not, I think it'd be easier
just to ping back the PerformanceEntry, since it can be easily processed
along with the rest of the entries on the timeline.

My thinking with the circular buffer was that if the developer was really
interested in something, they would grab it. And if they haven't grabbed it
by the time the buffer was full, they probably didn't care. I guess this is
mainly a question for long-running sites. Would they be more likely to wait
for the buffer to be full and then pick out the ones they want? Or would
they rather monitor for an important resource and then record it as soon as
they see it?

I also kind of liked the idea of letting measures be multi-phase operations
just like resources and the main document. Adding a bunch of marks to one
measure seems desirable to me, so that you can group together all the steps
in one place. Any thoughts on that one?

Thanks for coming up with this proposal!

James

On Thu, Jun 16, 2011 at 2:07 PM, Nic Jansma <Nic.Jansma@microsoft.com>wrote:

>  As discussed on yesterday's conference call, we see three primary changes
> with the Unified Timing v3 proposal:****
>
> ** **
>
> **1.       **A base object that all performance types inherit from.  The
> base object defines a few required attributes (eg., name, getDuration())**
> **
>
> **2.       **A circular buffer model that all performance types must
> participate in****
>
> **3.       **A restricted API set that performance types must be queried
> through (getEntryByType(), getEntryByName())****
>
> ** **
>
> We feel that #2 and #3 are significant changes to the behavioral models of
> the existing specs and may limit future development needs.  Jatinder has
> gone into some details of our concerns with these changes in his last email.
>
> We like #1, and think that it alone could open up new possibilities for a
> more unified interface.  What if we incorporated the idea of a base object
> type into our existing Navigation Timing, Resource Timing and User Timing
> specs that all performance entries must inherit from?  Then, we *add*getEntryByType() and getEntryByName() to the API set we have today to allow
> for the "playbook" scenario, where you can easily query all performance
> types at once and place them on a single timeline.  We can remove any
> redundant methods that have been replaced by getEntryByType() and
> getEntryByName(), e.g., getResourceTimings().  Since they all inherit from
> the base class, they will all have a name, entryType, startTime, and
> duration associated with them that can be visualized.
>
> If we do this, any future performance entry that wishes to participate in
> this model simply inherits from the base type and be exposed through the
> unified getter, and since they're not restricted by #2 or #3, they could
> also implement any additional specialized APIs that they need.****
>
> ** **
>
> We think this integration meets the goals we have all discussed.****
>
> ** **
>
> Here's what the proposed updated IDL for all of the interfaces might look
> like, with significant changes bolded:****
>
> ** **
>
> interface Performance {****
>
> *    **// Unified API set***
>
> *    **const** **unsigned** **short** PERF_NAVIGATION = **0**;*
>
> *    **const** **unsigned** **short** PERF_RESOURCE = **1**;*
>
> *    **const** **unsigned** **short** PERF_DEVELOPER_MARK = **2**;*
>
> *    **const** **unsigned** **short** PERF_DEVELOPER_MEASURE = **3**;*
>
> * *
>
> *    PerformanceEntryList getEntriesByType(in **unsigned** **short**
>  entryType);*
>
> *    PerformanceEntryList getEntriesByName(in DOMString name**, **in
> optional **unsigned** **short** entryType); **// Added optional entryType*
> **
>
> ** **
>
>     // Navigation Timing****
>
>     readonly attribute PerformanceTiming timing;****
>
>     readonly attribute PerformanceNavigation navigation;****
>
> ** **
>
>     // Resource Timing****
>
>     const unsigned short INITIATOR_OTHER = 0;****
>
>     const unsigned short INITIATOR_LINK = 1;****
>
>     const unsigned short INITIATOR_CSS = 2;****
>
>     const unsigned short INITIATOR_SCRIPT = 3;****
>
>     const unsigned short INITIATOR_IMAGE = 4;****
>
>     const unsigned short INITIATOR_OBJECT = 5;****
>
>     const unsigned short INITIATOR_SUBDOCUMENT = 6;****
>
>     const unsigned short INITIATOR_XMLHTTPREQUEST = 7;****
>
>     const unsigned short INITIATOR_EMBED = 8;****
>
>     const unsigned short INITIATOR_AUDIO = 9;****
>
>     const unsigned short INITIATOR_VIDEO = 10;****
>
>     const unsigned short INITIATOR_SVG = 11;****
>
> ** **
>
>     void clearResourceTimings();****
>
> *    PerformanceResourceTimingList getResourceTimings();** // is now
> getEntriesByType(PERF_RESOURCE)***
>
> *
>     PerformanceResourceTimingList getResourceTimingsByURL(in DOMString url);
> ** // is now getEntriesByName(url, PERF_RESOURCE)***
>
>     void setResourceTimingBufferSize (in unsigned long maxSize);****
>
> ** **
>
>     attribute Function onresourcetimingbufferfull;****
>
> ** **
>
>     // User Timing****
>
>     const string MARK_FULLY_LOADED = "fullyLoaded";****
>
>     const string MARK_FULLY_VISIBLE = "fullyVisible";****
>
>     const string MARK_ABOVE_THE_FOLD = "aboveTheFold";****
>
>     const string MARK_TIME_TO_USER_ACTION = "timeToUserAction";****
>
> ** **
>
>     void mark(in DOMString markName);****
>
>     Array getMarks(in optional DOMString markName); *// see below******
>
>     void clearMarks(in optional  DOMString markName);****
>
> ** **
>
>     void
>  measure(in DOMString measureName, in optional DOMString startMark, in optional DOMString endMark);
> ****
>
>     Array getMeasures(in optional DOMString measureName);* **// see below*
> *****
>
>     void clearMeasures(in optional DOMString measureName);****
>
> };****
>
> ** **
>
> interface PerformanceEntry {****
>
>     readonly attribute DOMString name;****
>
>     readonly attribute unsigned long long startTime;****
>
>     readonly attribute unsigned short entryType; ; // A PERF_* constant***
> *
>
>     readonly attribute unsigned long long duration; // may be 0****
>
> }****
>
> ** **
>
> interface PerformanceEntryList {****
>
>     readonly attribute unsigned long length;****
>
>     PerformanceEntry item(in unsigned long index);****
>
> }****
>
> ** **
>
> interface PerformanceNavigation {
> // does not inherit from PerformanceEntry****
>
>     const unsigned short TYPE_NAVIGATE = 0;****
>
>     const unsigned short TYPE_RELOAD = 1;****
>
>     const unsigned short TYPE_BACK_FORWARD = 2;****
>
>     const unsigned short TYPE_RESERVED = 255;****
>
>     readonly attribute unsigned short type;****
>
>     readonly attribute unsigned short redirectCount;****
>
> };****
>
> ** **
>
> interface PerformanceTiming : *PerformanceEntry* {****
>
>     readonly attribute unsigned long long navigationStart;****
>
>     readonly attribute unsigned long long unloadEventStart;****
>
>     readonly attribute unsigned long long unloadEventEnd;****
>
>     readonly attribute unsigned long long redirectStart;****
>
>     readonly attribute unsigned long long redirectEnd;****
>
>     readonly attribute unsigned long long fetchStart;****
>
>     readonly attribute unsigned long long domainLookupStart;****
>
>     readonly attribute unsigned long long domainLookupEnd;****
>
>     readonly attribute unsigned long long connectStart;****
>
>     readonly attribute unsigned long long connectEnd;****
>
>     readonly attribute unsigned long long secureConnectionStart;****
>
>     readonly attribute unsigned long long requestStart;****
>
>     readonly attribute unsigned long long responseStart;****
>
>     readonly attribute unsigned long long responseEnd;****
>
>     readonly attribute unsigned long long domLoading;****
>
>     readonly attribute unsigned long long domInteractive;****
>
>     readonly attribute unsigned long long domContentLoadedEventStart;****
>
>     readonly attribute unsigned long long domContentLoadedEventEnd;****
>
>     readonly attribute unsigned long long domComplete;****
>
>     readonly attribute unsigned long long loadEventStart;****
>
>     readonly attribute unsigned long long loadEventEnd;****
>
>   ****
>
> *    **// name: "document"***
>
> *    **// entryType: PERF_NAVIGATION***
>
> *    **// startTime**:** navigationStart (keep navigationStart as well
> since the attribute is available in UAs today)***
>
> *    **// duration: loadEventEnd - navigationStart***
>
> };****
>
> ** **
>
> interface PerformanceMark : *PerformanceEntry* {****
>
> *    **// name: mark's name***
>
> *    **// entryType: PERF_DEVELOPER_MARK***
>
> *    **// startTime: mark time***
>
> *    **// duration: always 0***
>
> }****
>
> ** **
>
> interface PerformanceMeasure : *PerformanceEntry* {****
>
> *    **// name = measure's name***
>
> *    **// entryType: PERF_DEVELOPER_MEASURE***
>
> *    **// startTime: measure's start time***
>
> *    **// duration: measure's duration***
>
> }****
>
> ** **
>
> interface PerformanceResourceTiming : *PerformanceEntry* {****
>
>     readonly attribute unsigned short initator;****
>
> ** **
>
>     readonly attribute unsigned long long redirectStart;****
>
>     readonly attribute unsigned long long redirectEnd;****
>
>     readonly attribute unsigned long long fetchStart;****
>
>     readonly attribute unsigned long long domainLookupStart;****
>
>     readonly attribute unsigned long long domainLookupEnd;****
>
>     readonly attribute unsigned long long connectStart;****
>
>     readonly attribute unsigned long long connectEnd;****
>
>     readonly attribute unsigned long long secureConnectionStart;****
>
>     readonly attribute unsigned long long requestStart;****
>
>     readonly attribute unsigned long long responseStart;****
>
>     readonly attribute unsigned long long responseEnd;****
>
> ** **
>
> *    **// name: url renamed to 'name'. this is the resource URL***
>
> *    **// entryType: PERF_RESOURCE***
>
> *    **// startTime: resourceStart renamed to 'startTime'***
>
> *    **// duration: responseEnd - startTime***
>
> };****
>
> ** **
>
> *Note:****
>
> Our thoughts were to keep getMarks() and getMeasures(), even though they
> can be satisfied by getEntriesByType().  They can provide a simpler return
> value than getEntriesByType():****
>
> mark("a");****
>
> mark("a");****
>
> mark("a");****
>
> mark("b");****
>
>  ****
>
> getMarks();****
>
> // returns {a:[1,2,3],b:[4]}****
>
>  ****
>
> getEntriesByType(performance.PERF_DEVELOPER_MARK);****
>
> // returns [{name:"a",startTime:1,entryType:2,duration:0},****
>
> //          {name:"a",startTime:2,entryType:2,duration:0},****
>
> //          {name:"a",startTime:3,entryType:2,duration:0},****
>
> //          {name:"b",startTime:4,entryType:2,duration:0}]****
>
> ** **
>
> - Nic****
>
> ** **
>
> *From:* public-web-perf-request@w3.org [mailto:
> public-web-perf-request@w3.org] *On Behalf Of *Jatinder Mann
> *Sent:* Tuesday, June 14, 2011 7:09 PM
> *To:* public-web-perf@w3.org
> *Subject:* RE: [Resource Timing] Unified Timing v3****
>
> ** **
>
> Thank you James for putting the time into formulating this proposal. As per
> action items from the last conference call, we have been actively evaluating
> this proposal.****
>
> ** **
>
> You are right that the current specification for Navigation, Resource and
> User Timings has a drawback that all APIs are exposed on window.performance.
> Your goal with the unified model is to attempt to give structure to the APIs
> and make them future proof – which are excellent goals. However, as we have
> evaluated this proposal, we have found some major drawbacks:s****
>
> ** **
>
> ***1.      ****Generalized vs. specialized.*
>
> The proposal attempts to create a single interface that all Timings must
> use for data collection, storage and retrieval. At first glance, the unified
> interface looked like a very good idea. However, we are concerned that this
> single interface constrains current and future Timings to be generalized and
> risks losing, or making it difficult, to have access to functionality
> specific to that Timing. ****
>
> ** **
>
> For example, we lose the ability to do custom filtering of data querying.
> With the Unified Timing API, there is no way to do custom UA-based filtering
> of the data, which can do the filtering more efficiently than script, as we
> limit ourselves to just getEntriesByType() and getEntriesByName(). Let’s
> consider a fictional future Widget Timings API. Imagine a future WidgetEntry
> that fires 1000s of times a second, yet one of the common ways of
> interacting with it is to only get Widgets of a specific "size".  Instead of
> having to return 1000s of objects back via getEntriesByType(), it would be
> nice to allow for getWidgetsBySize(500) or getWidgetsWithSizeBetween(50,
> 100). The generalized API has made this task difficult by requiring that the
> complete results be returned by getEntriesByType() by iterating over the
> entire set, at a loss of efficiency. Further, if the WidgetEntry only has
> "duration" and "size" attributes, but is logistically linked to a "parent"
> Widget and we don't put that "parent attribute" as part of its WidgetEntry,
> there would be no way to loop over all of the widgets to find children of a
> specific parent. The generalized API has made this task difficult. A
> specialized API would allow us to do something as simple as
> getWidgetsByParent(foo).****
>
> ** **
>
> While this is just one fictional example we're portraying, we think a
> generalized interface will introduce these types of complexities and
> limitations.****
>
> ** **
>
> **2.       ***Single, circular buffer. *****
>
> This model is heavily structured around a single circular buffer. However,
> this has a few downsides. ****
>
> ** **
>
> **a.       **Items stored in the buffer. Navigation Timing is a singleton
> and Developer Timing is a pay per use model; do either of these need to be
> stored in a buffer? What if Graphics Timing stores frames per second data;
> does this go in the same buffer?****
>
> ** **
>
> **b.       **Circular buffer and fullness. The buffer is defined as a
> circular buffer, which is different from the capped/limited buffer discussed
> in Resource Timing.  The advantages of a circular buffer is that there is no
> need to monitor the "fullness" of it, but there are disadvantages.  Since
> the items aren't stable within the buffer, sequential querying of the
> interface can lead to inconsistent results if the developer isn't aware of
> the circular behavior.  Consider this naive way of using the interface:
>
> var theFirstResource = getEntriesByType(PERF_RESOURCE)[0];
> // the 151st resource is downloaded, and kicks the first entry out since
> it's a circular buffer
> var theSecondResource = getEntriesByType(PERF_RESOURCE)[1]; // INCORRECT --
> this is actually the 3rd downloaded resource as the buffer wrapped
>
> On the other hand, with the capped buffer in Resource Timing, no external
> events modify the state or order of "current" items in the array, unless the
> user explicitly clears the buffer (clearResourceTimings()).****
>
> ** **
>
> **c.        **clearBuffer() clears the entire buffer.  What if I’m not
> only interested clearing Marks and not Resource Timing or Navigation Timing?
> ****
>
> **d.       **Default buffer size of 150 was designed for the estimated
> average number of resources on a future page. With Unified Timing, this
> buffer size requirements will limit memory growth of all types. Consider the
> case of Widgets Timing, where Widgets are added 1000 times every second.
> While Widgets are not enabled by default per your suggestion, developers may
> want to enable them because they're very common.  The circular buffer will
> mean previous Navigation and Resource Timing data is lost immediately. This
> forces the user agent to either specify a large buffer size, which is a
> waste of memory, or the developer must increase the buffer size in the
> <head>, which is a poor performance practice, or the developer must let data
> be lost, which is a poor experience. None of these options are attractive.
> ****
>
> ***3.      ****Additional concerns:*
>
> **a.       **Unified Timing's getEntriesByName() could have collisions
> amongst the various types.  For example, ResourceTiming is keyed by an URL.
> If we add a future timing type that is keyed by URL, there would be a
> collision.  Developers querying the interface would have to be careful to
> check the returned timing type.****
>
> **b.       **Disabling new Timing types by default. Considering adding
> script to the <head> is a poor performance practice and analytics libraries
> will want timing enabled by default, disabling Timings by default is not the
> best approach. ****
>
> **c.        **The "onentryadded" event isn't filtered by type, so if we
> have future high-frequency timing events, the event could introduce overhead
> if script is just looking for a specific subsets of events.  Dealing with
> the various timing types will require lots of "if typeof" statements.****
>
> **d.       **The Unified APIs make querying the data a bit more verbose. Consider
> querying the results of NavigationTiming, ResourceTiming and UserTiming:**
> **
>
> performance.timing****
>
> performance.getResourceTiming()****
>
> performance.getMarks()****
>
> ** **
>
> The Unified APIs is a bit longer to read:****
>
> performance.getEntriesByType(performance.PERF_NAVIGATION)[0];****
>
> performance.getEntriesByType(performance.PERF_RESOURCE);****
>
> performance.getEntriesByType(performance.PERF_DEVELOPER_MARK);
>
> Since the Unified API might kick timings out of the buffer, just querying
> for navigationStart changes from:****
>
> var navStart = performance.timing.navigationStart;****
>
> ** **
>
>            To:****
>
>               var navs =
> performance.getEntriesByType(performance.PERF_NAVIGATION);****
>
>               if (navs[0]) {****
>
>                      var navStart = navs[0].navigationStart;****
>
>               }****
>
> * *
>
> I think we need to step back and try to understand what kind of an API a
> web developer would find useful that allows for specialized methods and at
> the same time gives a consistent structure. For example, a hierarchy under
> the window.performance namespace may preserve both structure and
> specialization of the API. Eg., performance.navigation,
> performance.resource, performance.developer, performance.graphics would
> contain all APIs for Navigation Timing, Resource
>