Re: Progress on test harness

On Fri, 30 Mar 2007 15:22:53 +0200, Dominique Hazael-Massieux <dom@w3.org>  
wrote:

>
> Hi,
>
> A few progress to report on the test harness:
>  * the list of test cases is now taken from the database, rather than
> hardcoded in the script
>  * I've made it possible to associate specific HTTP headers with a given
> test case in the process; this was already needed for one test suite I
> wanted to integrate in the harness
>  * the SVG Tiny test suite is now available through the harness:
> http://www.w3.org/2007/03/mth/harness ; I wonder if I should also
> include SVG Basic; none of my mobile devices has SVG support, so I
> haven't recorded any useful results at this time.
>  * at each step you have now indications of where you are in the test
> suite
>
> I haven't implemented the session id concept yet, nor its related
> features.
>
> I'm also trying to figure out whether I can include the OMA Test suites
> in that harness within their licensing terms.
>
> As a reminder, the results table are linked from:
> http://www.w3.org/2007/03/mth/results
>
> Dom

Dom:

During last week's meeting, we discussed testing the DOM test suite  
through the harness, and I promised to send Allan a mail about my proposal  
such that he could figure out possible solutions, but I decided playing  
with it a little myself. This is what I've come up with.

It would be of advantage to allow for automatic testing, such that a test  
suite can be run autonomously. My approach uses JavaScript for that.
The form element inserted by the harness would need a name, e.g. <form  
name="testevaluation">. Then any script can trigger the result submission  
by itself.
Example: In the file selfhtml.js in the 1. test of the DOM test suite, the  
function setResult writes the result to the screen, based on the variable  
resultType. Simply adding to it:
'document.testevaluation.result[0].click()' in the pass conditional  
section,
'document.testevaluation.result[1].click()' in the fail conditional  
section,
'document.testevaluation.result[2].click()' in the else conditional section
would trigger the report of the current test result (and, of course, then  
the outcome doesn't have to be written on the screen anymore). So, the  
test harness needs a name for the form element, and the result evaluating  
script would have to be modified a bit. As a consequence, all DOM/JS tests  
could be run automatically.

-- 
Till Halbach
Quality Assurance, Opera Software (www.opera.com)

Received on Tuesday, 3 April 2007 12:13:18 UTC