Re: Takeaways WPT & ARIA test harness

Hi Alistair,

Good ideas. Agree on the general best practices

Some comments i would like to add.

Accessibility testing uses functional scenarios to navigate the features 
but specifically targets to verify the WCAG SC
Accessibility test steps can have a logical flow dependency, for example, 
The Page has a non empty page title,  The title is meaningful.
The pass / fail would be the results of all the logical tests passing or 
any one step failing. 
The test could be automated, semi automated and manual.


Warm regards,

Charu Pandhi
---------------------------------------
Accesibility Tooling and Automation lead
IBM Accessibility Research
 (512) 286 6370, T/L 363 6370
http://w3.ibm.com/able   http://www.ibm.com/able

 
      Vice President, SWE Southwest Texas
      swt.swe.org





From:   Alistair Garrison <alistair.garrison@ssbbartgroup.com>
To:     Wilco Fiers <wilco.fiers@deque.com>, "public-wcag-act@w3.org" 
<public-wcag-act@w3.org>
Date:   10/05/2016 06:27 AM
Subject:        Re: Takeaways WPT & ARIA test harness



Hi, 
 
From a quick bit of research this week – sorry.
 
Accessibility testing is effectively an extension of User Acceptance 
Tests—in effect, ascertaining if a product (e.g., web site or web 
application) enables users, including persons with disabilities—to 
complete all specified tasks, whether are using Assistive Technology (AT) 
or not.
 
Accessibility tests are generally written to ensure positive accessibility 
features have been implemented or that negative features have been 
avoided. Ideally, features are defined in terms of testable statements 
that can be made about the product. 
 
·         Positive feature: A skip link is provided at the top of a page 
that goes directly to the main content area. 
·         Negative feature: No displayed content in a page flashes more 
than three times per second.
 
As such, accessibility tests can be written following the same 
well-established software testing best practices that we use to write 
general acceptance tests, for example:
 
1)            When covering a feature, write a test procedure that 
contains several simple test steps rather than a few complex test steps.
2)            Ensure each test step checks for one thing.
3)            Write each test step in plain English (avoiding 
pseudo-code).
4)            Ensure each test has an expected result that is clearly 
defined.
5)            Ensure each test is independent. It should be able to run on 
its own, not affect other tests if changed, and not be affected by changes 
to other tests. 
 
Section 11 of IEEE 829-2008 contains details of environmental needs that 
should be documented – for example, hardware or software needed, etc.
 
Very best regards
 
Alistair
 
From: Wilco Fiers <wilco.fiers@deque.com>
Date: Wednesday, 5 October 2016 at 10:23
To: "public-wcag-act@w3.org" <public-wcag-act@w3.org>
Subject: Takeaways WPT & ARIA test harness
Resent-From: <public-wcag-act@w3.org>
Resent-Date: Wednesday, 5 October 2016 at 10:24
 
Hi team, 
I've updated our wiki page with some research I did about WPT & ARIA. I 
think there are some interesting ideas here for us to consider.
 
https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Testing_Resources#Take-aways_from_WPT_.26_ARIA_Test_Harness

 
-- 
Wilco Fiers - Senior Accessibility Engineer