[minutes] Web Performance WG Teleconference #103 2013-03-27

Meeting Summary:



1.     Asynchronous Scrolling

In this week's conference call, we discussed the asynchronous scrolling feature. This feature was motivated by developers trying to benchmark their site by repeated calls to scrollTo() and rAF(). The goal of this feature is to help developers test that their page scrolls and renders smoothly. The first aspect of this feature, providing an API for performance smooth scrolling as the user does via the UI, is tracked by the CSS WG. The Web Perf WG is interesting in finding a way to report a metric to the user on the scrolling performance. At this time, we haven't closed on what that metric will be, but this was a good introduction discussion and we intend to continue the investigation and discussion on the mailing list and possibly future calls.



2.     Topics for next conference call

Next week we will be spending the entire time discussing the diagnostics and error logging feature. We have a draft of the Error Logging specification, https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/ErrorLogging/Overview.html, please review it prior to next week's call.




W3C Web Performance WG Teleconference #103 2013-03-27



IRC log: http://www.w3.org/2013/03/27-webperf-irc



Meeting Minutes: http://www.w3.org/2013/03/27-webperf-minutes.html



Attendees

James Simonsen, Nat Duca, Tom, Jason Weber, Matt Kotsenas, Jatinder Mann, Daniel Austin, Ganesh Rao, Alois Reitbauer, Rob Dickenson,


Scribe

Jatinder Mann



Agenda

1.     Discuss Asynchronous Scrolling

--------------------------------------------------------------------------------



Minutes:
Asynchronous Scrolling
James: We have found that developers are interested in measuring scrolling performance. We have seen poor patterns that use scrollTo and rAF to measure scrolling performance. We have also had web developers reach out to us to ask how to get this information.
... There are two real issues here: a way to do a smooth scrolling instead of a quick scroll, and the second is to get the measurement data. The first one should be address by CSS WG that is working on a smooth scrolling version of scrollTo. What I was really hoping was that we define the measurement data.
Tom: I can help by giving use cases. About a month ago Pintrest, and we helped them with the usual suspects on our end, like DropShadows and Hover Effects. What they struggled with was how to measure the improvements. One data point we did look at was frame rate. We saw the frame rate improve from 30FPS to 60FPS.
James: Another use case is something like how slowest is my slowest frame rate, in the janky / hanging case.
Jason: We don't necessarily seeing the problem in rendering, but more often in the user's JavaScript code and what the system is trying to do on top of that to keep scrolling smooth. What we're hopping to do is fix that issue on the system end so developers don't need to update their code patterns.
Tom: Is the goal that developers will not be able to introduce jank using their code? Seems like in the near future we will continue to have developers shoot themselves in the foot and be able to jank.
Jason: I see the value in developers measure their performance on a device basis. I'm not sure what that metric would look like however. For example, you may have a moment in which a JS event fires that causes a jank. This can be both percieved jank where there was visual change in the page that looks like a jank, but isn't necessary a jank. Other cases you can real jank.
Nat: In Chrome, we see the same problem. We have a rendering thread that's running at 60 frames, but their is a visual jank where it appears that the scroll is jerky.
Jason: With IE10 on Windows 8, we draw more than a screensize to make sure there is a degree of smoothing scrolling
Nat: In Chrome's Impl Site Painting, we are doing more of that where we are rendering more than the screen size. But in some cases we will checkerboard, in other cases you will see a jank.
... In some cases, you can opt out of this. In Jansa's demo, you can see that they have opt'd out to see how jank free they can keep their scrolling.
Tom: If our concern is that there are multiple threads running and we really want to tell developers how free is the JS thread or how jank free it is, do we want to provide that informaiton and let them interpret as they would? How much time was the thread free in a time period? Like thread utilization.
Jason: I see this more like thread frequency. Like if your render thread drops its frequency, you can tell there was a difficult graphics draw. We could also give information on the frequency of the individual frame times of the DOM and JS threads.
... There are two things that I struggle with and that's why we haven't made this available. First, there is just so much information that we would provide and it's not clear how to make this information actionable to real users. For example, in the Pinterest example, it's not clear how the frequency will help them make actionable changes.
... There are also cases where there is a real jank or percievable jank. For example if your scrolling, the UI thread could have had an operation that took a couple hundred milliseconds, whereas the rendering and other threads are moving in the normal frequency, where you would have jank. Not sure if that UI frequency is useful.
Nat: I like your first point. We have a lot of arguments of actionability. We haven't until now even brought this up on this point. What made us bring this up to this group, is that we wanted to developer more data for RUM. Getting data for real user monitoring is useful.
Tom: I think the low bar of giving metrics that site owners can monitor to see that at least their performance has regressed with their new version. The data may not be useful in making actionable decisions, but at least they will know that there is a performance problem. The thing I worry about is that things may look fine on a developer class machine and real world testing shows problems.
Nat: Looks like we agree that there is a problem, we just need to figure out what it would be.
Jason: We are open to considering doing something here, but we're just not sure what that metric would be. We are open to suggestions.
James: We can't expose instananeous number and expect JavaScript to monitor it. Instead we may want to include some sort of timing data on this in the Performance Timeline.
Jatinder: Would this be something that would need to be enabled rather than always on. Something we had discussed in the diagnostics tracing type thing.
James: That's what I was thinking.
Nat: One of the problem is that jank may be defined different by different people. In talking to thirty people, we have different definitions of jank.
Jason: We consider jank as the draw rate is falling below the frame rate of the display.
Nat: How do you take care of idleness?
Jason: You essentially throw out the idle information.
... That's not something you would do in javascript, that's something you would do in the browser.
Nat: We do have kind of information internally as well.
Jason: We have similar information internally.
Tom: Are is just behind a flag in Chrome, so it's not available by default, but can be turned on.
Nat: Do you have a notion of vsync on the threads?
Jason: We have a notion of vsync on the threads.
Nat: Which would be the same value as rAF?
Jason: That's just for JavaScript. We actually schedule all of our work around vsync to get more efficient batching and coalesing and let the CPU get to more lower power state.
Nat: In our architecture, we have two frame counters and drop counters. Not sure what the others do. Some older browsers may only have a single JS thread.
... We actually have three threads as well. We call ours Browser compositor. We have two compositors. If you have a red div and turn it blue. It'll first go to the CSS compositor thread, and then it'll go to the browser compositor which sits in our sandbox and then it'll go to DirectX.
Jason: You really want parallelization early in the process.
Nat: XHRs and timers on our platform run whenever, they don't schedule on vsync. But other things run on vsync.
Jason: That's something we would love to see you guys align with the vsync, that will definitely help you be more power efficient for your customers.
James: Have we thought about including this in the timeline?
Jatinder: We have always talked about putting more information on the timeline, so I think that's fine. But let's first find out what kind of informationwe want to put on the timeline.
Jason: There are two types of janks, from touch and mouse.
Nat: We treat all of our touch and mouse the same way. Browser to render thread and render thread rejects keyboard then it goes to fast or slow path. The render thread can paint the entire display tree.
Nat: There is some stuff about dev docs on chromium. We're pretty open about that. You're right that the low end devices is the problem here. I think we should just let this be for now and bring this topic back up when we come up with a good option. Looks like the options are either a counter or a timeline.
Jason: We want to be sure the data is forward looking.
Diagnostics and Error Logging.
Jatinder: Seeing that we only have fifteen minutes left, I recommend that we schedule a full hour discussion next week on Diagnostics and Error Logging. I have posted a first draft of the error logging spec on the mailing list. Please review for discussion next week.