Re: Automated and manual testing process

Mike,
The discussion is hitting on the issues I’m concerned about, absolutely.

In our SC requirements (https://www.w3.org/WAI/GL/wiki/WCAG_2.1_Success_Criteria) we include "Be testable through automated or manual processes” and I think that we need to make sure that we have a common understanding of that to proceed efficiently.

Thanks,
AWK

Andrew Kirkpatrick
Group Product Manager, Standards and Accessibility
Adobe

akirkpat@adobe.com
http://twitter.com/awkawk


From: Mike Elledge <melledge@yahoo.com<mailto:melledge@yahoo.com>>
Reply-To: Mike Elledge <melledge@yahoo.com<mailto:melledge@yahoo.com>>
Date: Monday, January 30, 2017 at 11:15
To: "White, Jason J" <jjwhite@ets.org<mailto:jjwhite@ets.org>>, David MacDonald <david100@sympatico.ca<mailto:david100@sympatico.ca>>, "Bailey, Bruce" <Bailey@access-board.gov<mailto:Bailey@access-board.gov>>
Cc: WCAG <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Subject: Re: Automated and manual testing process
Resent-From: WCAG <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Resent-Date: Monday, January 30, 2017 at 11:18

Hi all--

This is an interesting, useful, and thought-provoking thread, but I do have a more general question:

Are we answering your question, Andrew? Is there a particular circumstance we should be addressing?

My two cents...

Mike


On Monday, January 30, 2017 11:00 AM, "White, Jason J" <jjwhite@ets.org<mailto:jjwhite@ets.org>> wrote:




From: David MacDonald [mailto:david100@sympatico.ca]

If we can establish a metric that is reliable and that we can establish a high "inter rater reliability" among human testers who "derstand how people with different types of disabilities use the Web" then it is testable. In that case it's not "subjective" in the eyes of WCAG, and can be formulated as a testable statement.
[Jason] This is a superb statement of the position that, as I recall, was developed during the evolution of WCAG 2. The challenge is to write the requirements in a way that facilitates achieving the best inter-rater reliability that we can (unfortunately without the benefit of a formal study to verify and improve it, unless someone volunteers to run such experiments).


________________________________
This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.

Thank you for your compliance.
________________________________