RE: Ethics of user testing and people with cognitive disabilities

Hi Josh,

It's important to keep in mind that "cognitive disabilities" is a wide spectrum. On the low end are people with intellectual disabilities, and on the high end are all of us as we age. In between are many people, such as those with Dyslexia, Discalculia, etc. The decision-making capacity of this population concomitantly varies widely.

My opinion is that people with cognitive disabilities should be trusted to make their own decisions. In the little time we have with them for studies, we cannot competently judge how well they make decisions. Thus we should rely upon them to be competent to make decisions, just as we would for anyone else, unless there is compelling evidence to the contrary. For example, in the U.S., people judged incompetent to make decisions have legal guardians who can do so for them or, better yet, with them.

To help successful decision making, explanations must be simple to understand. Other factors help as well, such as patience and repetition if needed. As we all know, such practices are good for everyone.

One ancillary but important point is that, at least in the U.S., people with cognitive disabilities are protected by institutional review boards, which must approve every facet of a study before it is undertaken. This is a result of a history of taking advantage of people with cognitive disabilities, such as tricking them into agreeing to participate in harmful studies. That alone justifies your wise question.

John

John Rochford
Director, INDEX Program
Instructor, Family Medicine and Community Health
Eunice Kennedy Shriver Center
University of Massachusetts Medical School
www.DisabilityInfo.org

Please excuse typos and brevity. Message composed on mobile device.

Confidentiality Notice:
This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential, proprietary, and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender immediately and destroy or permanently delete all copies of the original message.
-------- Original message --------
From: josh@interaccess.ie
Date: 7/11/16 6:14 AM (GMT-05:00)
To: w3c-wai-gl@w3.org, team-wcag-editors@w3.org
Subject: Ethics of user testing and people with cognitive disabilities

[Chair hat off]

In a previous thread the issue of user testing with people with cognitive impairments has been brought up. I thought this could be a good time to
share some of my own thoughts in this area. As some of you may know, I ran a user testing lab in the National Council for the Blind of Ireland for around 10 years. During that time I ran user tests with a wide range of users, not just those who are blind/VIP. In that time I did a small amount of testing with people with cognitive impairments, mental health issues etc. I found this a difficult group to test with due to my own concern about the ethics of doing so properly.

One very strong reservation I have about this whole area is simple. Does the user have the ability the objectively separate the tasks they are  asked to perform in a test (and the natural success/failure when trying to completing these tasks) from their overall 'sense of self'? What I mean is that will the user be able to realise that their actions are being objectively observed without any 'judgement' on their performance?

 I would hate to think that a user would come away from a user test, where many tasks were failed (which is great usability information) but feeling worse about themselves, or as if _they_ were some kind of failure. In short, I think user testing is a bit of a performance, within an utterly contrived environment. Some people take to this well, others don't.

This is a very thorny issue but one I want to flag. I don't think testing should take place at all without a strong framework about how to deal with these sensitive situations. Sometimes you may have to make a call not to test, if it isn't in the best interest of the user test participant, even if they may be a perfect candidate for 'rich data'. I've made this call not to test in the past, and my overall take away was that I am largely uncomfortable doing this kind of testing, unless I'm sure it is in the participants best interest, separately from whatever the secondary need of a project/client is.

Thoughts?

Josh