W3C home > Mailing lists > Public > public-web-perf@w3.org > September 2012

RE: Tests returning an inconsistent number of results

From: Karen Anderson (IE) <Karen.Anderson@microsoft.com>
Date: Tue, 18 Sep 2012 22:53:01 +0000
To: James Graham <jgraham@opera.com>, "public-web-perf@w3.org" <public-web-perf@w3.org>
Message-ID: <748A9FD1BD28E74F971C8D69F634947217BFD6E2@BL2PRD0310MB361.namprd03.prod.outlook.com>
We started down this path with Navigation Timing with checkin #170.  At the time we wanted to avoid scripting errors when a UA had not fully implemented part of a spec while running the tests and made it such that you didn't have to put every call within an individual try/catch block.  What did you have in mind?

-Karen

-----Original Message-----
From: James Graham [mailto:jgraham@opera.com] 
Sent: Thursday, September 13, 2012 1:31 AM
To: public-web-perf@w3.org
Subject: Tests returning an inconsistent number of results

Several test submissions here use a design where if the feature is not available they fail to run the rest of the testsuite (e.g. [1]). This is an antipattern because it makes it unnecessarily difficult to compare the results between builds; ideal tests should always return the same number of results.

Will anyone object if I patch the problematic tests and commit the fixes?

[1]
http://w3c-test.org/webperf/tests/submission/Google/HighResolutionTime/test_cross_frame_start.html
Received on Tuesday, 18 September 2012 22:54:13 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 18 September 2012 22:54:13 GMT