W3C home > Mailing lists > Public > public-test-infra@w3.org > April to June 2012

Re: How to keep test results for separate products?

From: Linss, Peter <peter.linss@hp.com>
Date: Thu, 31 May 2012 15:34:09 +0000
To: Jeanne Spellman <jeanne@w3.org>
CC: "<public-test-infra@w3.org>" <public-test-infra@w3.org>
Message-ID: <EDFA7A25-A5D4-4224-8BB6-E09D8037E213@hp.com>
On May 30, 2012, at 1:14 PM, Jeanne Spellman wrote:

> I have been running some sample tests for ATAG and quickly realized that I have no way to save the results with the name of the authoring tool being tested.  It looks to me as if the way I will have to use the test harness is to keep a separate spec-name/data for each product being testing and have the manifest file return to the same test suite location.
> 
> Since I haven't worked with the submitted/approved directory structure yet, I don't know the impact of what you are proposing.
> 
> I would like to ask your recommendations of the best way to manage testing different authoring tools both web-based and non-web-based (e.g. Wordpress, Blogger, Dreamweaver, InDesign, Word, Drupal, etc).  We need the harness to present the instructions to the tester, record the results and produce pass/fail reports by authoring tool.
> 
> 

Hi Jeanne,

this is an interesting twist, the framework was designed for testing user agents, not authoring tools. It presumes a static test and multiple viewers. 

One way to handle this is to use the multiple format support of the framework, making one 'format' for each authoring tool. The framework then needs to get a per-suite switch to break out results by format instead of user agent.

Let me give this some more thought on how best to handle itů

Peter
Received on Thursday, 31 May 2012 15:38:16 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 31 May 2012 15:38:16 GMT