Goals

Help screen readers render ARIA as well as browsers render HTML and CSS.

Method

Co-author screen reader test plans for aria practices with screen reader developers . Develop a test runner and test format to collect data on these test cases in a standard way, and create a plan for completing test runs at a regular cadence.

The phases below are sequential. The work streams within a phase can happen in parallel.

PHASE 1 (Q1)

Overview: Collect feedback from screen reader vendors on the initial test design, assertion model, and their future involvement in the project. Create wireframe for test runner.

Work stream 1.1: Collect screen reader developer input with demo testing for checkbox, combobox, menubar, and grid

Timeline Task Owner Collaborators
Feb 3 Week 1 Build tool for reviewing existing tests of a design pattern Bocoup
Feb 10 Week 1-2 Complete test files for checkbox, combobox, menubar and grid ARIA-AT CG Bocoup
Feb 17 Week 3 Test these patterns with JAWS, NVDA, and VoiceOver in Chrome and Firefox on Windows and Chrome on Mac in the ARIA-AT prototype runner. Upload test results into ARIA-AT repository. ARIA-AT CG Bocoup
Feb 24 Week 4 Refine reports of test results Bocoup
March 2 Week 5 Share test results with NVAccess, Freedom Scientific, and Apple Matt King Bocoup, AT developers, ARIA-AT CG
March 9 Week 5 Collaborate to create a plan for addressing ongoing feedback on runner design, test design, assertion design and results reporting ARIA-AT CG Bocoup
March 16 Week 5-6 Adjust test and assertion design based on feedback Bocoup
March 16 Week 5-6 Collaborate to create a plan for ongoing project participation in test writing, including:
  • How/when to review test plans and results for each pattern
  • How/where to provide feedback on test plans
  • and results
  • Github policies
ARIA-AT CG Bocoup, AT developers
March 23 Week 7 Share project vision, viability, plan and stakeholder contribution options with a wider audience. ARIA-AT CG Bocoup, TBD

Work stream 1.2: Create design mock-ups for runner website base on experience with prototype

Timeline Task Owner Collaborators
Feb 3

Feb 10

Week 1-2 Collaborate and document requirements for the test runner. Analyze gaps between prototype and use cases Bocoup ARIA-AT CG
Feb 17

Feb 24

Week 3-5 Create wireframes for test runner and results websites Bocoup

PHASE 2 (Q1-Q2)

Overview: Create the test runner and the results website. Collect feedback and address concerns. Create a test contribution workflow. Run all tests and report any bugs found in browsers, accessibility APIs and assistive technologies.

Work stream 2.1: Build production system

Timeline Task Owner Collaborators
Week 1 Research and plan:
  • Break work into milestones based on use case prioritization
  • Develop high-level system requirements for each milestone
  • Break work into sprints for each milestone
ARIA-AT CG Bocoup
Week 2-7 Develop production system Bocoup
Week 7 Provide feedback on testing system AT developers Bocoup
Week 8-12 Develop production system Bocoup
Week 13 Prioritize feedback from pilot test run of production system: ARIA-AT CG Bocoup
Week 13-15
  • Fix critical issues identified by pilot
  • Final production system prep
Bocoup ARIA-AT CG

Work stream 2.2: Write tests and run pilot before end of development phase

Timeline Task Owner Collaborators
Week 1-10 Write test plans for baseline set of APG patterns ARIA-AT CG
Week 10-12 Review test plans with screen reader developers ARIA-AT CG
Week 12 Train testers ARIA-AT CG Bocoup, testers
Week 13 Run pilot tests of baseline set
  • each baseline test run by a TF member and a trained tester
  • compare TF member results with tester results, look for system problems, usability problems, training problems.
Testers Bocoup
Week 13-15 Complete development of test plans ARIA-AT CG
Week 15+ Run first complete round of tests with 6 browser/screen reader combinations ARIA-AT CG