RE: [requestAnimationFrame] Test Suite

Hi Boris,

Thank you for bringing that to my attention.  I know we discussed it at some point but didn't realize it made it into any of the specs.  I see that http://www.w3.org/TR/animation-timing/#the-WindowAnimationTiming-interface shows cancelAnimationFrame, whereas http://www.w3.org/TR/2011/WD-animation-timing-20110602/#the-WindowAnimationTiming-interface shows cancelRequestAnimationFrame.  From what I can tell, of the browsers that have implemented the cancel functionality, they have done so with cancelRequestAnimationFrame.  I will update the test to use cancelAnimationFrame.  

WRT the test against 60Hz, I think we can all agree that the base case need for this API is to fire callbacks at the same rate as the user's vsync (with some allowed variance for load etc).  That is what this test intends to do.  60Hz seems to be the most common refresh rate.  If that's not true, then we should change the test to what is.  If that means that when a user is running at 60Hz that the test fails and they need to adjust their settings and run it again, so be it.

What I pushed was not a definitive list of everything that can be tested.  Certainly more tests around the processing model could be beneficial.  However, these are conformance tests.  We want to be interoperable, but testing what isn't conformance does not belong in a W3C test suite.  The processing model states:

5.For every context in contexts, in any order, perform the following steps: 
	1.Let d be context's active document.
	2.If d's hidden attribute is true, continue to the next entry in the contexts list. Otherwise proceed with these steps.
	3.Let doclist be d's animation frame request callback list.
	4.Append all entries from doclist into list preserving order.
	5.Clear doclist.

Note that there is not a callback order guarantee between contexts, only within a given context.  Certainly having a test for callback order guarantee within a browsing context makes sense.

Thank you for the feedback.  We can discuss more in today's call.
-Karen

-----Original Message-----
From: Boris Zbarsky [mailto:bzbarsky@MIT.EDU] 
Sent: Tuesday, February 28, 2012 2:03 PM
To: public-web-perf@w3.org
Subject: Re: [requestAnimationFrame] Test Suite

On 2/28/12 2:47 PM, Jatinder Mann wrote:
> Karen has submitted the requestAnimationFrame test suite to the 
> submission folder. Please take a moment to review the test suite and 
> provide feedback.

The tests should be testing cancelAnimationFrame, not cancelRequestAnimationFrame, right?

There should probably be more extensive testing of the processing model. 
  Since I know for a fact, based on previous discussion on this list, that what WebKit and IE implement in this regard is quite different, if they're both passing the tests that means the tests are not testing enough of the "interesting" cases.

Maybe that's OK; I doubt those cases actually matter in practice.  But maybe it's not if we think we'll ever want actual interop there.

Things that probably need additional testing are at least ordering of callbacks within and across documents and behavior on cancellation of a callback from inside another callback.  I'm pretty sure Gecko would fail that last one right now, for example.

> We have verified that IE10 and Chrome pass the test suite. Boris, can 
> you check Firefox?

Whether Firefox passes depends on various external factors (machine load, etc), because of the timing-dependent part of the test.  Of course the same is true for Chrome when I just tested it, and I wasn't even testing with any particular load on the processor...

I'm not sure what normative requirement those tests are testing in the first place, of course.  The only support for the "should fire at 60Hz" 
bit seems to be an informative note.  Those tests should not be part of a conformance test suite for this feature, imo, though they can be part of an optional "quality of implementation" test if someone wants.

Firefox passes all the non-timing-dependent tests.

-Boris

Received on Wednesday, 29 February 2012 17:41:11 UTC