Re: Is there an official place to document current role compatibility differences?

On Apr 9, 2014, at 4:55 AM, Karl Groves <kgroves@paciellogroup.com> wrote:

> 
> On Tue, Apr 8, 2014 at 11:00 PM, James Craig <jcraig@apple.com> wrote:
>> On Apr 8, 2014, at 1:46 PM, lisa.seeman <lisa.seeman@zoho.com> wrote:
>> 
>>> By the way...I also put up an online database were people can add unit tests and document failure and successful use of ARIA. Very different from the test harness as this is looking for authoring errors rather then operating system/ browser implemetion errors. I dont see anyone using it yet, but it is at http://athena-ict.com/UnitTestsHomePage.html.
>> 
>> I think this illustrates why the work should be done in a W3C source repository, even if it is a community effort.
> 
> As it currently stands, the PFWG test harness URL you mentioned:
> 
> 1. Returns 404 when attempting to access the main test suite URLs
> 2. Requires a W3C account to access any of the other links

ARIA is moving from private space to public (hence this list). The test harness source files (test) are already public. The viewable interface of the test harness should be public as well, but there may be some pages where that is not apparent, or the authentication is still mistakenly required. The editing interface requires an account for obvious reasons. I assume the proposed approach for maintaining your database would also require an account of some sort, so I don't see how this is different. 

> The effort John (and others) started is to have completely open access to contribute/ view individual tests, to do the testing itself, and (eventually) to the data that it generates.    As Bryan noted, there is a risk of PEBKAC influencing the results, but I hold optimism that the crowdsourced nature of the testing will reduce the noise there.
> 
> All content, tests, results, literally *everything* about the Open Accessibility Testing project is MIT licensed and anyone and everyone who wishes to contribute can and should do so.  We invite everyone to contribute test cases as described at https://github.com/Open-A11y-Testing/Test-Triage/blob/master/contributing.md
> 
> Also, to address your concerns about bugs: I feel it would be wrong not to report bugs. As an effort aimed at trying to make the Web more accessible, we'd be missing that goal if we didn't use that data to submit issue reports to browser vendors and AT vendors.

Thank you for understanding.

> To the extent possible, it'd be good for browser and AT vendors to help this by documenting the process of submitting issue reports.  Perhaps as a document in the "Test Site" repository? https://github.com/Open-A11y-Testing/Test-Site

Good idea. I already listed the WebKit bug search and new bug report URL in your issue tracker. Perhaps you could stub out what you have in mind (TBD sections listed by user agent?) and I'll try to fill in any aspects that are relevant to WebKit. I assume David, Alex, Cynthia and others would be willing to fill in their parts of a document like that, too.

One more piece of advice is to try to make the test cases complete. We had a bunch of ARIA test cases originally that were invalid uses of ARIA. For example <div role="gridcell"></div> is out of context and will not be considered a gridcell, so it's not a valid test case. Even <div role="grid"><div role="row"><div role="gridcell"></div></div></div> is invalid for several reasons: It's empty, it's only one cell with no header cells, etc. 

The best test case examples are ones that can be automated, or at least ones that are fully functional, such as this tree control. https://dvcs.w3.org/hg/pfwg/raw-file/default/ARIA/1.0/tests/test-files/_functional/tree/ariatree2.html which tests tree and related roles, aria-activedescendant, aria-expanded, level computation, label calculation, etc. Functional tests are harder to create, but they are most useful.

Cheers,
James

Received on Wednesday, 9 April 2014 19:15:21 UTC