W3C home > Mailing lists > Public > public-web-perf@w3.org > September 2012

Re: Tests returning an inconsistent number of results

From: Philippe Le Hegaret <plh@w3.org>
Date: Wed, 19 Sep 2012 15:42:07 -0400
To: James Graham <jgraham@opera.com>
Cc: public-web-perf@w3.org
Message-ID: <1348083727.12930.1073.camel@chacal>
On Thu, 2012-09-13 at 10:31 +0200, James Graham wrote:
> Several test submissions here use a design where if the feature is not 
> available they fail to run the rest of the testsuite (e.g. [1]). This is 
> an antipattern because it makes it unnecessarily difficult to compare 
> the results between builds; ideal tests should always return the same 
> number of results.
> 
> Will anyone object if I patch the problematic tests and commit the fixes?

Hi James,

you're welcome to patch and commit. If we run into issues, we can always
revert if needed,

Philippe

> [1] 
> http://w3c-test.org/webperf/tests/submission/Google/HighResolutionTime/test_cross_frame_start.html
> 
> 
Received on Wednesday, 19 September 2012 19:42:18 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 19 September 2012 19:42:18 GMT