RE: 答复: W3C Navigation Error Logging - log when UA crash/freeze?

I agree it’s hard differentiate if it’s the code/computer/browser/plugin that caused the problem. What I really expect to be useful is the change in rate of crashes during a deployement of new site content. In our case, it was particularly a bug in handling style sheet commands that caused the UA to crash, and it wasn’t all UA’s, it was version specific. But it was something we didn’t find until the code was released past the small flight phase. Enough users had to notice it and manually report it and then for us to see the trend in reports.

The whole goal of Nav Error Logging, at least in my mind, is to get all of the bad things that happen to end users back to the site operators so they can be aware of problems and ultimately improve the experience for end users. And that needs to occur automatically, not manually. Crashing and frozen browsers are probably the most damaging of broken experiences possible for end users. While they will be more rare than HTTP 500’s, I hope, they are worst case scenarios.

My actionable scenario plays out like:


1.       Normal monitoring of Nav Errors is collecting the background noise rate of crashes for your site.

2.       You rollout your feature to the test in production bed and flight it for a few percent of users.

3.       Overall flight metrics look good, no obvious perf regression and crash rate is within normal limits.

4.       Start ramping up the new feature to 100% of users.

5.       As the rollout crosses about 35% of user, you detect an increase of crash reports coming back in Nav error telemetry, and the reported URLs are highly correlated with the new features URLs.

6.       Stop the rollout, investigate, fix, rinse repeat ……..

I see this as no different than the goal of getting back when a user sees an HTTP 500 error.

Aaron


From: Ilya Grigorik [mailto:igrigorik@google.com]
Sent: Wednesday, October 22, 2014 8:29 AM
To: Liang,Tao
Cc: Aaron Heady (BING AVAILABILITY); public-web-perf@w3.org
Subject: Re: 答复: W3C Navigation Error Logging - log when UA crash/freeze?

Aaron, Liang, do you have any existing data on size of the problem? Examples of how and where it helped?

I figure every browser maintains some internal tracking for crash counts (we do with Chrome), but it's not clear to me that this data is actionable for the developer. Many crashes have nothing to do with site's application code, and often times its hard to figure out who's at fault (site vs. browser bug, etc).

ig

On Wed, Oct 22, 2014 at 1:58 AM, Liang,Tao <liangtao01@baidu.com<mailto:liangtao01@baidu.com>> wrote:
I agree with Aaron Heady's advice. In fact, it's useful to see the release's effect on crash for global user base and to supply a way to improve the product.
But does Navigation have an opportunity to logging Crash / Breeze ?


-----邮件原件-----
发件人: Aaron Heady (BING AVAILABILITY) [mailto:aheady@microsoft.com<mailto:aheady@microsoft.com>]
发送时间: 2014年10月22日 4:25
收件人: public-web-perf@w3.org<mailto:public-web-perf@w3.org>
主题: RE: W3C Navigation Error Logging - log when UA crash/freeze?

Any thoughts on this?

-----Original Message-----
From: Aaron Heady (BING AVAILABILITY)
Sent: Tuesday, October 14, 2014 1:38 PM
To: 'public-web-perf@w3.org<mailto:public-web-perf@w3.org>'
Subject: W3C Navigation Error Logging - log when UA crash/freeze?

Curious about a possible addition to the scope of what logging covers. Does anything in the spec capture when a page load crashes the browser tab or causes it to be unresponsive? We ran into an obscure situation that we hadn’t tested for and managed to get a UA to crash, looks like it was related to a style recalculation.

Could we/should we log an error entry on the page/domain when the UA crashes/freezes, so we can see at scale that something we released is increased the crash/freeze rate for the global user base?

Thoughts?

Aaron