RE: [RequestAnimationFrame] clarification of what 'time' is

I had a talk with some folks here at Microsoft to go over this and the suggestions were very similar to Nat’s ideas. The quick back of the hand idea suggested the RAF could have additional arguments for skipped frames, last vsync and next vsync. Very similar ideas.

From: nduca@google.com [mailto:nduca@google.com] On Behalf Of Nat Duca
Sent: Wednesday, April 15, 2015 10:46 AM
To: Jerry Jongerius
Cc: Todd Reifsteck; Michael Blain; public-web-perf; Justin Rogers; Tobin Titus
Subject: Re: [RequestAnimationFrame] clarification of what 'time' is

I agree that the current timestamp we pass into raf per the processing model leads to inferior animations.

But I'm not convinced that we should change the current timestamp definition. I think this is a much deeper problem possibly warranting a raf level 2 effort.

The problem here is that there are many different timestamps for a frame, and depending on which type of animation you are doing and what effect you desire, you may want to use a different one of those timetsamps...

Eg:
{
   frameBeginTime:
   frameDeadline: ...
   deltaTime: ...
   outputTime: ...
}

I believe deltaTime is roughly what the current processing model is trying to allow authors to compute. I believe this is most useful for hacky animations that are based on exponetial decay of something [new = curX * f(decay,deltatime) + some inverse]. For those, you can't really interpolate on a timestamp value, so you want to just know how much elapsed since last time.

Some other animations really want a stable frameBeginTime. This is what jerry refers to. This is an important timestamp for when you start animations, for instance, I think?

Output time is yet another interesting timestamp. This is important when you want something to appear to stay under the finger. The output time is the time when we predict the frame will appear on glass. If you have an input event that was received at ts=10, and you know the pipeline is 60ms deep, then ideally one forward predicts your animations to the output time. Its also the number that recovers best when you drop a frame --- the output time typically keeps incrementing by 16.6ms (etc) every frame, even when the rendering system is double-framing in order to catch up.

Finally, deadlines are useful for knowing how much work you can do before you blow the frame budget. Its kind of like saying the display refresh rate, which isn't always 60hz, but subtracting out known browser overheads. For instance, chrome typically needs 4ms per frame, so on a 120hz display, your frame budget is actually 4ms. [Though, setIdle/setImmediate may be better here].


Possibly, a raf level 2 would be something designed with fantastic scripted animations in mind from the getgo. Perhaps it passes in a dict instead of position arguments with ambiguous semantics.

Though, perhaps I'm thinking too hard. It happens often. :D

On Wed, Apr 15, 2015 at 5:36 AM, Jerry Jongerius <jerryj@duckware.com<mailto:jerryj@duckware.com>> wrote:

Todd,



In a nutshell: To eliminate micro-jitter in animations driven by the ‘time’ argument – which are ultimately caused by (well known; see vsynctester.com<http://vsynctester.com>) delays in invoking the RequestAnimationFrame Processing Model.



If inter-frame times are charted, they are not flat (they should be), but jump around, sometimes a lot (by 30% of the frame interval is not uncommon in IE, along with phase shifting).  Even though those rendered frames are then presented to the end user VSYNC aligned (by OS level compositing; an example is DWM in Windows).  Because of this, any time offset from true vsync passed into RequestAnimationFrame only adds unnecessary changes to the location of objects that moved in an animation.



In other words, on my system, rendered frames are presented to the screen every 16.721 ms.  My animation code could care less about the 0ms to 4ms internal web browser delay in calling RequestAnimationFrame, because the rendered frames are presented to the screen perfectly aligned on the next vsync interval.  However, if objects in my animation were actually placed in a frame based upon rAF time argument “vsync+jitter”, instead of “vsync+0”, then all objects that moved in that frame will have unnecessary micro-jitter in their placement.



For example, what if my time driven animation is: scroll an image by ‘ms’ pixels every rAF, where ‘ms’ is the inter-frame time?  As the spec is written today, the image might very well be scrolled by these pixel amounts:



18.021

15.421

19.221

14.221

17.621

15.821



even though these rendered frames are then presented to the end user every 16.721 ms.  When clearly, every rAF, the image should be been scrolled by this number of pixels every frame:



16.721

16.721

16.721

16.721

16.721

16.721



- Jerry



PS: The HTML animation example in the spec does not work in IE




From: Todd Reifsteck [mailto:toddreif@microsoft.com<mailto:toddreif@microsoft.com>]
Sent: Wednesday, April 15, 2015 3:18 AM
To: Michael Blain; Jerry Jongerius
Cc: public-web-perf; Justin Rogers; Tobin Titus
Subject: RE: [RequestAnimationFrame] clarification of what 'time' is

Jerry/Mike
Could you quickly explain the scenario that this update would improve?

Thanks,
Todd

On Tue, Apr 14, 2015 at 10:29 AM, Jerry Jongerius <jerryj@duckware.com<mailto:jerryj@duckware.com>> wrote:
The 'time' in the rAF callback (Processing Model) should be altered in the
spec to allow (encourage) a web browser to pass the known begin time of the
frame (the vsync timebase) as the 'time', rather than performance.now().