RE: Proposal for new version of Requirement 3.7 Motivation

I like the idea "we could consider pairing aspects of the usability testing sessions with tangible results or improvements that came directly from this testing. That way, the testing outcomes and related improvements could be linked to specific methods for instance, or techniques or what not, and we could measure just how many of the improvements came directly from involving end users with disabilities in the overall process."
We should try to go beyond the technical methods and should include some guidance on the quality processes to be followed. Not all the findings of user testing can always be incorporated due to various technical and organizational factors. Therefore, the organizations can be rewarded for documenting the findings of the user testing and the subsequent actions  they took to improve the accessibility of their product. The documentation can help the organization to clearly assess their process maturity level. It will also help adhering to the WCAG standards in a more systematic way.

From: Denis Boudreau [mailto:denis.boudreau@deque.com]
Sent: Tuesday, April 9, 2019 7:58 AM
To: Detlev Fischer <detlev.fischer@testkreis.de>
Cc: public-silver@w3.org
Subject: Re: Proposal for new version of Requirement 3.7 Motivtion

I like the proposal with Chuck’s edits.

I disagree with your position Detlev, but understand your concerns. The temptation to game the system would undoubtedly rise from some of the people out there that would want to be able to claim a quick path to success (oh yeah, we tested with people, and “they” said it was fiiiiiiine...).

I’m just not able to agree with a statement such as:

“[testing]... does not in itself change the quality of the site under test. An awful site stays awful even after a lot of user testing.”

I believe that conducting testing with people with disabilities, when done genuinely with the goal of user experience improvements does absolutely change the quality of the site under test. The findings brought up by consulting those users is expected to bring forth positive changes. An awful site is supposed to get better as a result of the change that come from the activity of involving those users in the process. That’s just the nature of the activity. But we need a way to measure that clearly in Silver.

I celebrate our vision of rewarding usability testing with end users with disabilities. It does expose our model to abuse - I certainly share Detlev’s concerns here - but I’m sure that as we get to defining the details of how the scoring system will pan out, we’ll find ways to reward usability testing for aspects that actually provide value, not for things that pay lip service to the idea of making the product or service accessible.

As an example, we could consider pairing aspects of the usability testing sessions with tangible results or improvements that came directly from this testing. That way, the testing outcomes and related improvements could be linked to specific methods for instance, or techniques or whatnot, and we could measure just how many of the improvements came directly from involving end users with disabilities in the overall process. The more improvements came out direct end users contributions, the higher the points.


/Denis

—
Denis Boudreau
Principal accessibility SME & Training lead
Deque Systems, Inc.
514-730-9168



On Tue, Apr 9, 2019 at 04:30 Detlev Fischer <detlev.fischer@testkreis.de<mailto:detlev.fischer@testkreis.de>> wrote:
As I have said before, I think the mere fact that testing with users
with disabilities has taken place should not be rewarded since it does
not in itself change the quality of the site under test. An awful site
stays awful even after a lot of user testing. If then, as a result of
such testing, the accessibility and/or usability is improved, that
should impact also the conformance to measurable criteria (whether
absolute or score-based) - and I am happy to see those criteria extended
to realms so far difficult to measure.

Am 08.04.2019 um 20:42 schrieb Jeanne Spellman:
> Here is the proposal for revision of Requirement 3.7 Motivation as
> requested by AGWG to make it measureable.
>
> Motivation
>
> The Guidelines motivate organizations to go beyond minimal
> accessibility requirements by providing a scoring system that rewards
> organizations that demonstrate a greater effort to improve
> accessibility.  For example, Methods that go beyond the minimum (such
> as: Methods for Guidelines that are not included in WCAG 2.x A or AA,
> task-completion evalations, or testing with users with disabilities)
> are worth more points in the scoring system.
>
>
>

--
Detlev Fischer
Testkreis
Werderstr. 34, 20144 Hamburg

Mobil +49 (0)157 57 57 57 45

http://www.testkreis.de<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.testkreis.de&data=02%7C01%7CSSAHASRABUDHE%40ets.org%7C903b55c752544a3ba41c08d6bce2d0c5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636904079699835430&sdata=gg%2F3WMQzvPG2yzgxHbAQ%2FCUjg6qgEJd%2BbXS%2Bgsx27XU%3D&reserved=0>
Beratung, Tests und Schulungen für barrierefreie Websites

--
/Denis

--
Denis Boudreau
Principal SME & trainer
Web accessibility, inclusive design and UX
Deque Systems inc.
514-730-9168

Keep in touch: @dboudreau

________________________________

This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.


Thank you for your compliance.

________________________________