RE: Is there an official place to document current role compatibility differences?

Hi Bryan
The test harness is at http://www.w3.org/WAI/PF/testharness/

By the way...I also put up an online database were people can add unit tests and  document failure and successful use of ARIA. Very different from the test harness as this is looking for authoring errors rather then operating system/ browser implemetion errors. I dont see anyone using it yet, but it is at http://athena-ict.com/UnitTestsHomePage.html. 
Each unit test is added to a directory and in a page that uses frames so that people can add header scripts into the unit test. You can then run tools against it and see what they pick up (or not). feel free to add/use it as much as you want. 

All the best

Lisa Seeman

Athena ICT Accessibility Projects 
LinkedIn, Twitter





---- On Tue, 08 Apr 2014 22:44:42 +0300 Bryan Garaventa<bryan.garaventa@ssbbartgroup.com> wrote ---- 


Where can the ARIA test harness be accessed? This sounds like a good 
resource for conducting many of these tests. 
 
The standard user DB sounds good, but the problem with general user data is 
that it can often be sprinkled with user error data, such as the user simply 
not liking the intended interaction design of a widget, or the way that 
feedback is conveyed. I had that recently where a user said that ARIA Tabs 
were wrong, because he couldn't access them in the Links list. 
 
I always report bugs when I encounter something that I can prove is a bug, 
or reasonably so until proven otherwise, so no worries there. 
 
My main interest is to build out a single page high level overview resource 
that can be edited as needed to give a good overview of where the breakdown 
between browser support vs AT support differs, because then it will be 
easier to figure out where the main issues are located. I think this still 
will take a bit of playing with to get organized where version data can be 
included and variation notes can be added, since one way of doing things 
often varies greatly between another even if both ways are technically 
compliant. 
 
Apologies if I haven't replied earlier to others, my main comp died and Win 
is being reinstalled, so I'm using a backup for the time being. 
 
 
-----Original Message----- 
From: James Craig [mailto:jcraig@apple.com] 
Sent: Tuesday, April 08, 2014 11:40 AM 
To: Foliot, John 
Cc: Birkir Gunnarsson; Bryan Garaventa; Cynthia Shelly; public-pfwg@w3.org 
Subject: Re: Is there an official place to document current role 
compatibility differences? 
 
On Apr 8, 2014, at 11:25 AM, Foliot, John <john.foliot@chase.com> wrote: 
 
> I think we are talking past each other. There won't be "a list" - there 
will be a data-store of test results of ARIA attributes (and combinations 
when required) tested against [Operating system + versions] + [Browser + 
versions] + [Assistive Technology + versions]. It will all be captured in a 
single database - the end. 
 
It sounds a lot like the ARIA test harness. Why are you duplicating this 
effort instead of contributing to the existing one? We've got 700+ test 
cases already. 
 
> No, asking the crowd to be that granular is a recipe for low 
participation. 
 
At a minimum then, the tool should not prevent them from entering this 
critical information. 
 
> Once the data store starts growing, it would be relatively easy for an 
engineer at any company to run a query to ferret out bugs in their current 
builds 
 
Browser and AT manufacturers deal in bug numbers of thousands or tens of 
thousands with every release and have work flows revolving around the 
official bug databases for those products. It's naïve to assume an engineer 
would manually compare and resolve the results of one database query against 
another. If your database included a place for a bug number, plus other 
useful information such as "last tested version", this would not have to be 
manual.