Re: CFC: Manual testing processes

Wayne wrote:
> I think there may be cases where user testing is necessary.

That isn't the question. The question is whether usability testing should be mandated as part of showing WCAG 2.1 conformance (including for passing exceptions).

The rest of this email explores why, but the process issue is that it wasn't part of WCAG 2.0, so a major change to approach will probably not pass muster anyway.

Also, this email is focusing on the usability-accessibility overlap and how that affects organisations, I think there is another discussion to be had on personalisation.


> I think it is time for accessibility auditors to include user testing when no other reliable testing method exists to assess the presence of a barrier.

I would dispute that usability testing will be in any way reliable for answering the sort of questions that the guidelines would pose. The questions are very specific, about how people interpret certain things, and in the context of a website  it will be very difficult to remove other factors, or even ask the right questions to show what you need.

To be frank, we run usability testing every week, almost daily, and currently a small proportion of that is with people who have disabilities. It would be great for my business if usability testing were required.

However, I think the overall impact would be negative because less organisations would do *anything* for accessibility because the perceived (and in some cases actual) cost would be too high.

The balance that WCAG has achieved so far is keeping the overhead "reasonable". That is critical for the success of the guidelines and it has meant that governments are happy to point to the guidelines as the source material. If WCAG 2.1 suddenly costs 4 times as much to implement, businesses will lobby Governments to avoid it somehow, whether that is by specific exclusions or simply not using it at all. That will apply to any other accessibility guidelines to.

I've argued elsewhere [1] that some of the new SCs essentially require a UX process. Without that you have no basis of determining what is important to the user.

Some organisations do have the resources and process in place where they already do UX type work, but by no means all.

That is why I would be in favour of Lisa's WCAG AA+ idea, where there is a group of more process-oriented guidelines that are applicable when you have a general public remit, such as Governments and national companies. I don't think WCAG should specify who, that's a policy decision. Organisations that are small, or have niche audiences would not have to, but it is there for the general public, larger scale sites.

Whether that is a WCAG 2.1 thing I don't know, but it seems like the best avenue pre-WCAG 3.0.

Kind regards,

-Alastair

1] https://github.com/w3c/wcag21/pull/95#issuecomment-279971636

Received on Thursday, 16 February 2017 10:10:40 UTC