Re: [RequestAnimationFrame] What about paint-callbacks finishing too late for display-refresh

In reply to myself:

Actually it doesn't need not be the callback return that determines when 
the frame is drawn; rendering might happen async in the background, for 
example in WebGL, so can extend past the time the callback returns.
So if the callback returns before the display-refresh, the rendering 
might still be finished too late for it.

So the actual issue is:

"What about *frame-drawing* finishing too late for display-refresh?"

Now wildly assuming the frame would just be displayed at the first 
possible display-refresh after it's done rendering, this would emphasize 
another issue: It is unknown beforehand when a frame is actually 
displayed. Which, if not tackled somehow, would prevent smooth 
animations. Frame-drawing code not knowing for which moment in time 
ahead to draw the frame, results in well-known artifacts like jumps and 
stuttering.

A good solution is, for a given animation-sequence, to lower the 
animation-frame-rate to a constant divider of the actual 
display-refresh-rate. The divider is determined by the maximum number of 
display-refreshes any frame-drawing took, recently. This way, everything 
is still running vsync, while even the longest lasting frame-drawings 
fit in the custom frame-interval. Buffer-swaps of shorter-lasting frame 
drawings are delayed to align the same frame-interval. This way the 
moment of display is known beforehand again, allowing for proper animation.

Only when a new maximum frame-draw refresh-span is reached a 'sync 
error' (slight hick-up) might be visible. (But some determination 
(calibration) of the max refresh-span might even be possible to do 
'invisibly' beforehand.)

So we need a way to *delay the display* of the shorter lasting 
frame-drawings so they align to the chosen frame-interval. The actual 
display is in the realm of the user-agent, there's no way yet to 
reliably influence that in user-code, so we need an extension in the 
API: A return-value of the callback-function could be supported, 
specifying how many extra display-refreshes need to be *skipped* before 
actually displaying. ('undefined' would default to zero).

This is most flexible and keeps options open for users to program their 
own adaptive frame-rate housekeeping. (or create nice little libraries 
for that).

Additionaly some support could be baked-in for the most common cases.
For example, some lightweight Animator object, which increases/decreases 
the refresh-skips to fit the max encountered frame refresh-span during 
some recent period. It would be possible and advisable to users to use 
different Animator objects if they expect different frame-drawing 
workload characteristics, thus frame-rates, for different parts of their 
animations. (Even simultaneously).
'requestAnimationFrame()' would be a method of Animator. (maybe simply 
named 'requestFrame()' in that case)
(Or perhaps the registration of the animation-callback-function should 
stay active until unregistered, and not need to be issued for each 
frame. Furthermore, it only needs to be one, a delegate.)

Another advantage of such an Animator object dialogue is that a method 
or callback-param like 'lastFrameTime' could be supported (we need this 
;-), since it is clear to which frame callback we are referring (in 
contrast to a Window scoped implicit Animator having multiple different 
clients/callback functions running at the same time). It provides a 
clear, animation-scoped 'dialogue'.
On the other hand, when callback-functions would stick to the 
Window-object until deregistered, such multi-callbacks spanning 
dialogues could be supported as well, but that would change programming 
style (negatively perhaps) a bit.

Using something like the Animator keeps the measurement of a suitable 
frame-interval out of the user's hands. However, since different 
strategies and definitions for a 'suitable' frame-interval exist, and 
depend on the user's intents, this should perhaps be an additional, not 
the only way of controlling framerate. The Animator could just as well 
be implemented as user-code.

So far my thoughts, I may diverge completely from you intentions, and 
browser reality, but perhaps this aligns or comes to use in one way or 
the other.

Regards,

Jan Linnebank




On 16-7-2011 4:51, Jan Linnebank wrote:
> Regarding "Timing control for script-based animations, W3C Editor’s 
> Draft 27 June 2011"
>
> WHAT WITH PAINT-CALLBACKS FINISHING TOO LATE FOR SCHEDULED 
> DISPLAY-REFRESH
>
> To the editors,
>
>
> What would happen when a user's frame-callback doesn't return in time 
> for the frame to be actually displayed on it's scheduled display-refresh?
>
> Will it just be displayed on a later display-refresh?
>
> Will the user have access to the timestamp of that actual later 
> display-refresh (perhaps as extra param in the next callback, e.g. 
> 'lastRefreshTime')?
> Or will that timestamp simply be determinable by subtracting the 
> refresh-interval from the timestamp of the next requestAnimationFrame 
> callback?
> (The latter two questions motivated again with regard to psychological 
> reaction-time related experiments.)
>
> Regards,
>
> Jan Linnebank
>
> at UvA NeuroTest B.V.

Received on Sunday, 17 July 2011 17:57:29 UTC