Re: Test Framework Task Force

Tobie,

I realize that your priority is automated testing of browsers, but the 
test framework would also be helpful to other working groups who may not 
need the sophistication and automation, but whose testing work would be 
greatly facilitated by a testing framework tool.  Whatever you build 
into the test framework, please take into consideration some of the 
needs for manual testing, or at least, not preclude them, so that they 
could be added later.

I have been eager to use the testing framework for my working groups, 
but the current framework is not oriented toward the more manual testing 
needs of many accessibility guidelines and the non-browser testing of 
the Authoring Tool Accessibility Guidelines (ATAG).

Although the wiki page is marked as out of date, the requirements of the 
WAI domain are still relevant.
http://www.w3.org/wiki/Testing/accessibility
Since that wiki page was created, I have experimented with the
framework, and have the following requests to make the framework
useful to AUWG and UAWG:

1) the ability to test and report by product, not by browser  (e.g.
conformance of Dreamweaver, Blue Griffin, MediaWiki, Wordpress, etc.)

2) the user interface and the reports meet WCAG 2.0 AA (the framework
will be used by current working group members with disabilities).

3) the framework allows import of specs that have letters in the section 
numbering (ATAG has Part A and B, which results in section numbers like 
A.2.3.1)

4) at minimum, the framework test harness has an additional answer
option of "not applicable" in addition to "skip" and the reporting
appropriately collects the different responses. Allowing the test
administrator to define the buttons for answering would be most helpful.

5) the framework test harness records the evaluator (person) name and
produce reports by evaluator

6) the framework test harness allows entry of text comments by the evaluator

Please feel free to contact me if you would like more context or have
better ideas for how we could test.

Regards,

jeanne

On 2/22/2013 1:26 PM, Tobie Langel wrote:
> Hi all,
>
> I've set up a home for the Test Framework Task Force[1] and a stub for its main deliverable[2].
>
> Like all other testing task forces we'll be using the public-test-infra@w3.org (mailto:public-test-infra@w3.org) mailing list. If you haven't already, please sign-up for it[3].
>
> I'll be working on the framework requirements over the week-end and into next week.
>
> Robin, how much of the existing framework do you think can be re-used? Is it worth it?
>
> Best,
>
> --tobie
> ---
> [1]: http://www.w3.org/wiki/Testing/Test_Framework_TF
> [2]: http://w3c.github.com/testing-task-forces/test-framework.html
> [3]: Just send an email to mailto:public-test-infra-request@w3.org
>
>
>
>
>

-- 
_______________________________
Jeanne Spellman
W3C Web Accessibility Initiative
jeanne@w3.org

Received on Wednesday, 20 March 2013 21:55:34 UTC