Re: [RequestAnimationFrame] Please just copy iOS CADisplayLink

If strictly following the iOS approach, at least one more thing is 
needed: a way to signal when a frame-drawing is ready for display. In 
iOS they have 'presentRenderBuffer() 
<http://developer.apple.com/library/ios/#documentation/OpenGLES/Reference/EAGLContext_ClassRef/Reference/EAGLContext.html>', 
in OpenGL libs this is done with a call to 'swapBuffers()' etc.
Such a method could be part of the Animator (or CADisplayLink in iOS 
speak).

However, this is perhaps unnecessarily low-level.
Maybe more convenient would be if we're able to say something like 'I 
want the frame I just drew to be displayed after <n> display-refreshes 
after the display-refresh of the last displayed frame . (The 
full-framerate case would thus have n=1).
That's similar to what I proposed in one of my previous mails, with the 
'skip-frames' return-value.

Regards,

Jan



On 17-7-2011 23:05, Jan Linnebank wrote:
> To the editors:
>
> I stumble upon the way iOS does things.
>
> This is just what we (everyone?) need. Nothing more, nothing less.
>
> Please just copy it and you're done. ;-)
>
> Almost identical to the Animator that I sketched. Just the right level 
> of abstraction. All low level primitives (to build upon), with higher 
> level state-handling for convenience.
>
> http://developer.apple.com/library/ios/#documentation/QuartzCore/Reference/CADisplayLink_ClassRef/Reference/Reference.html 
>
>
> Thank you,
>
> Jan
>
>
> On 17-7-2011 19:56, Jan Linnebank wrote:
>> In reply to myself:
>>
>> Actually it doesn't need not be the callback return that determines 
>> when the frame is drawn; rendering might happen async in the 
>> background, for example in WebGL, so can extend past the time the 
>> callback returns.
>> So if the callback returns before the display-refresh, the rendering 
>> might still be finished too late for it.
>>
>> So the actual issue is:
>>
>> "What about *frame-drawing* finishing too late for display-refresh?"
>>
>> Now wildly assuming the frame would just be displayed at the first 
>> possible display-refresh after it's done rendering, this would 
>> emphasize another issue: It is unknown beforehand when a frame is 
>> actually displayed. Which, if not tackled somehow, would prevent 
>> smooth animations. Frame-drawing code not knowing for which moment in 
>> time ahead to draw the frame, results in well-known artifacts like 
>> jumps and stuttering.
>>
>> A good solution is, for a given animation-sequence, to lower the 
>> animation-frame-rate to a constant divider of the actual 
>> display-refresh-rate. The divider is determined by the maximum number 
>> of display-refreshes any frame-drawing took, recently. This way, 
>> everything is still running vsync, while even the longest lasting 
>> frame-drawings fit in the custom frame-interval. Buffer-swaps of 
>> shorter-lasting frame drawings are delayed to align the same 
>> frame-interval. This way the moment of display is known beforehand 
>> again, allowing for proper animation.
>>
>> Only when a new maximum frame-draw refresh-span is reached a 'sync 
>> error' (slight hick-up) might be visible. (But some determination 
>> (calibration) of the max refresh-span might even be possible to do 
>> 'invisibly' beforehand.)
>>
>> So we need a way to *delay the display* of the shorter lasting 
>> frame-drawings so they align to the chosen frame-interval. The actual 
>> display is in the realm of the user-agent, there's no way yet to 
>> reliably influence that in user-code, so we need an extension in the 
>> API: A return-value of the callback-function could be supported, 
>> specifying how many extra display-refreshes need to be *skipped* 
>> before actually displaying. ('undefined' would default to zero).
>>
>> This is most flexible and keeps options open for users to program 
>> their own adaptive frame-rate housekeeping. (or create nice little 
>> libraries for that).
>>
>> Additionaly some support could be baked-in for the most common cases.
>> For example, some lightweight Animator object, which 
>> increases/decreases the refresh-skips to fit the max encountered 
>> frame refresh-span during some recent period. It would be possible 
>> and advisable to users to use different Animator objects if they 
>> expect different frame-drawing workload characteristics, thus 
>> frame-rates, for different parts of their animations. (Even 
>> simultaneously).
>> 'requestAnimationFrame()' would be a method of Animator. (maybe 
>> simply named 'requestFrame()' in that case)
>> (Or perhaps the registration of the animation-callback-function 
>> should stay active until unregistered, and not need to be issued for 
>> each frame. Furthermore, it only needs to be one, a delegate.)
>>
>> Another advantage of such an Animator object dialogue is that a 
>> method or callback-param like 'lastFrameTime' could be supported (we 
>> need this ;-), since it is clear to which frame callback we are 
>> referring (in contrast to a Window scoped implicit Animator having 
>> multiple different clients/callback functions running at the same 
>> time). It provides a clear, animation-scoped 'dialogue'.
>> On the other hand, when callback-functions would stick to the 
>> Window-object until deregistered, such multi-callbacks spanning 
>> dialogues could be supported as well, but that would change 
>> programming style (negatively perhaps) a bit.
>>
>> Using something like the Animator keeps the measurement of a suitable 
>> frame-interval out of the user's hands. However, since different 
>> strategies and definitions for a 'suitable' frame-interval exist, and 
>> depend on the user's intents, this should perhaps be an additional, 
>> not the only way of controlling framerate. The Animator could just as 
>> well be implemented as user-code.
>>
>> So far my thoughts, I may diverge completely from you intentions, and 
>> browser reality, but perhaps this aligns or comes to use in one way 
>> or the other.
>>
>> Regards,
>>
>> Jan Linnebank
>>
>>
>>
>>
>> On 16-7-2011 4:51, Jan Linnebank wrote:
>>> Regarding "Timing control for script-based animations, W3C Editor’s 
>>> Draft 27 June 2011"
>>>
>>> WHAT WITH PAINT-CALLBACKS FINISHING TOO LATE FOR SCHEDULED 
>>> DISPLAY-REFRESH
>>>
>>> To the editors,
>>>
>>>
>>> What would happen when a user's frame-callback doesn't return in 
>>> time for the frame to be actually displayed on it's scheduled 
>>> display-refresh?
>>>
>>> Will it just be displayed on a later display-refresh?
>>>
>>> Will the user have access to the timestamp of that actual later 
>>> display-refresh (perhaps as extra param in the next callback, e.g. 
>>> 'lastRefreshTime')?
>>> Or will that timestamp simply be determinable by subtracting the 
>>> refresh-interval from the timestamp of the next 
>>> requestAnimationFrame callback?
>>> (The latter two questions motivated again with regard to 
>>> psychological reaction-time related experiments.)
>>>
>>> Regards,
>>>
>>> Jan Linnebank
>>>
>>> at UvA NeuroTest B.V.
>>
>>
>>
>