W3C home > Mailing lists > Public > public-test-infra@w3.org > July to September 2011

Re: Automatic test results reporting

From: Linss, Peter <peter.linss@hp.com>
Date: Wed, 10 Aug 2011 11:14:46 -0700
Cc: Francois Daoust <fd@w3.org>, Philippe Le Hegaret <plh@w3.org>, public-test-infra <public-test-infra@w3.org>
Message-Id: <DA246B7A-127D-4742-B245-E407906D6354@hp.com>
To: Bryan Sullivan <blsaws@gmail.com>
The harness already has the capability to import results from a text file, this lets users gather results from other testing systems and submit them in a batch (it just takes an administrator to do the import currently, adding an upload page for users is on the todo list). 

Adding an API to accept results from a different system would be fairly trivial, my main concern would be adding some authentication to that API so that it can't get spammed. There's a user/login system growing under the hood (from common code used with the test suite management system), that could be used for that so only registered users can get to the API (or upload batch results).

Peter

On Aug 10, 2011, at 6:19 AM, Bryan Sullivan wrote:

> Hi Francois,
> 
> A useful feature would be to allow a tester to define the test results
> collection server address, and for W3C to publish the API that the client
> side of the framework will use to deliver the results. This will allow
> testers to do preliminary testing, and automatically collect the results at
> their own server for offline/private review, prior to doing a test for which
> results are delivered to W3C.
> 
> Selection of a non-default results server could be a configuration item
> which the tester can set at the start of a test session or persistently (eg
> via localstorage). 
> 
> Bryan
> 
> On 8/10/11 1:33 AM, "Francois Daoust" <fd@w3.org> wrote:
> 
>> Hi Philippe,
>> 
>> I've added an automatic test results reporting feature to the framework, see:
>>  http://w3c-test.org/framework/suite/nav-timing-default/
>> 
>> The page presents a "Report test results automatically (when possible)"
>> checkbox, enabled by default. When activated, pressing "start" will run the
>> test suite, jumping for test case to test case and saving the result reported
>> by "testharness.js".
>> 
>> That's a bit ugly from a pure visual point of view as the browser jumps from
>> page to page as soon as it gets the result of the test, but we can improve
>> that later on, I think.
>> 
>> Note automatic results reporting gets automatically deactivated when the user
>> chooses to report for a user-agent that is not the user-agent on which it runs
>> the tests.
>> 
>> Francois.
>> 
> 
> 
> 
Received on Wednesday, 10 August 2011 18:15:22 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 10 August 2011 18:15:22 GMT