Re: APA test harness availability

The PF test harness was *not* behind closed doors. There is way too much 
meme around that, and it is not constructive to the discussion.

The test cases and reports were all publicly visible, and continue to be so.

What required a login was creating or editing test cases, or entering 
test results. This is to make sure the people changing data in the tool 
were trained on its use, and so they could be credited / blamed for what 
was in the tool. This is an important feature, and common to many open 
services.

The login system of the tool allowed anyone with a W3C account to log 
in. This is a pretty broad access, since anybody can create an account 
for themselves.

In practice, we want mainly Working Group participants using the tool. 
This is so they are trained in its proper function, and know how to use 
the tool to support the Candidate Recommendation process, without 
putting in a bunch of stuff that's useful in its way but distracts from 
the CR work.

I feel like a broken record saying this, but if people want to 
participate in the work of the WG, they need to join the group. I don't 
see why this is such a problem that it keeps getting raised.

I'm sure there is indeed a role for people to create ARIA tests who 
don't have a reason to participate directly in the ARIA WG. That sort of 
thing is what the Test the Web Forward project was set up to do. The 
thing is, the ARIA test harness is an alpha-level tool, designed solely 
to do CR testing, and does not have the robustness to support widespread 
usage by people not closely connected to its goals. Because of the 
existence of better tools, it does not seem to be a good use of my time 
to try to bring the necessary robustness to this tool.

We've been talking for years about how to bring the features and test 
cases that are in the ARIA test harness to the Test the Web Forward 
project. I think that would be a better use of someone's energy who 
wants to focus on tests not directly related to the work of the Working 
Group. If people do want to participate in work directly related to the 
Working Group, they should join the group.

If people simply want to access the tests or reports for their own 
purposes, without recording data in the tool, that is already possible. 
If there are any bugs in that I will fix them if they are - politely - 
brought to my attention.

Michael

On 04/01/2016 10:01 AM, Léonie Watson wrote:
> Janina & Michael,
>
> The PF test harness and reports were behind closed doors. Will the APA test
> harness be available publicly for people to contribute to and/or access
> reports from?
>
> The information gathered through testing has enormous value beyond the WG.
> Help from the community in testing specs is also welcome, and the W3C test
> harness platform takes a lot of the hard work out of the equation when
> compared to other open platforms.
>
> Léonie.
>
>

Received on Monday, 4 January 2016 18:16:14 UTC