Re: [Test Harness] Additions to the test harness

On Mon, 24 Jan 2011, Anderson Quach wrote:

> 
> Hi public-html-testsuite,
> 
>  
> 
> In the Web Performance WG, we would like to integrate some additional common functionality into the common testharness.js
> that is being developed in the HTML WG [1]. This functionality currently resides under the resources folder under WebPerf[2].
> 
>  
> 
> These helper functions abstract the testing of Booleans, Equality and Greater than. We’d like to add the following helpers to
> testharness.js:

What is the use case for these functions? Do you have some example code? 
On the face of it it looks like the effect would be to have less code 
wrapped in callbacks to test() or test.step(). This defeats one of the 
core design goals of the framework which is to allow as many unexpected 
error conditions as possible to be handled gracefully (i.e. caught by a 
try/catch handler rather than propogated to the top level).

It is also unclear to me that the failed-asserts-are-fatal model will work 
well for performance testing. Indeed it seems possible that this is 
exactly the issue that you are trying to work around by adding these 
functions. Presumably for a performance test one typically wants all the 
results, even if one doesn't meet some expectations? Or am I 
misunderstanding how you intend to use the framework?