W3C home > Mailing lists > Public > public-html-testsuite@w3.org > January 2011

Re: [Test Harness] Additions to the test harness

From: James Graham <jgraham@opera.com>
Date: Tue, 25 Jan 2011 10:47:30 +0100 (CET)
To: Anderson Quach <aquach@microsoft.com>
cc: "public-html-testsuite@w3.org" <public-html-testsuite@w3.org>
Message-ID: <alpine.DEB.2.00.1101250942200.6155@sirius>


On Mon, 24 Jan 2011, Anderson Quach wrote:

> 
> Hi public-html-testsuite,
> 
> 
> 
> In the Web Performance WG, we would like to integrate some additional common functionality into the common testharness.js
> that is being developed in the HTML WG [1]. This functionality currently resides under the resources folder under WebPerf[2].
> 
> 
> 
> These helper functions abstract the testing of Booleans, Equality and Greater than. Wed like to add the following helpers to
> testharness.js:

What is the use case for these functions? Do you have some example code? 
On the face of it it looks like the effect would be to have less code 
wrapped in callbacks to test() or test.step(). This defeats one of the 
core design goals of the framework which is to allow as many unexpected 
error conditions as possible to be handled gracefully (i.e. caught by a 
try/catch handler rather than propogated to the top level).

It is also unclear to me that the failed-asserts-are-fatal model will work 
well for performance testing. Indeed it seems possible that this is 
exactly the issue that you are trying to work around by adding these 
functions. Presumably for a performance test one typically wants all the 
results, even if one doesn't meet some expectations? Or am I 
misunderstanding how you intend to use the framework?
Received on Tuesday, 25 January 2011 09:48:15 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 25 January 2011 09:48:16 GMT