Re: CFC: Manual testing processes

-1
Our testing methodology has not proven to be as reliable as we think. I
think there may be cases where user testing is necessary.

We are not making success criteria to make life easy for developers or
auditors. We are removing barriers.

We have a year and a half before this is released. I think it is time for
accessibility auditors to include user testing when no other reliable
testing method exists to assess the presence of a barrier.

There are reliable small sample testing techniques. Accessibility auditors
should learn them.

Wayne

On Mon, Feb 13, 2017 at 7:19 PM, Andrew Kirkpatrick <akirkpat@adobe.com>
wrote:

> Call For Consensus — ends Wednesday February 15th at 10:30pm Boston time.
>
> The requirements for WCAG 2.1 SC's (https://www.w3.org/WAI/GL/
> wiki/WCAG_2.1_Success_Criteria have been discussed at length. Included in
> the requirements is #2, which states "Be testable through automated or
> manual processes”, which indicates that in order for a success criteria to
> be regarded as “testable” it must be possible to determine whether a page
> passes that criteria using automated or manual testing processes.
>
> On last Tuesday’s call the WG came to a resolution regarding this item,
> specifically related to user testing. The group also surveyed this
> question, and arrived at a unanimous agreement:
>
> "User testing is not a required part of a manual testing process for WCAG
> test criteria.”
>
> This resolution indicates that if the only way to test a success criteria
> is to conduct user testing, then that is not “testable” with regard to
> WCAGT 2.1.
>
> The Working Group will recommend strongly in WCAG 2.1 (as it did in WCAG
> 2.0) that user testing be conducted.
>
> For background:
> Call minutes: http://www.w3.org/2017/02/07-ag-minutes.html
> Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results#xq1
>
> If you have concerns about this proposed consensus position that have not
> been discussed already and feel that those concerns result in you “not
> being able to live with” this decision, please let the group know before
> the CfC deadline.
>
> Thanks,
> AWK
>
> Andrew Kirkpatrick
> Group Product Manager, Standards and Accessibility
> Adobe
>
> akirkpat@adobe.com
> http://twitter.com/awkawk
>