Re: Proposal for new version of Requirement 3.7 Motivation

+1 to Denis and Shrirang.

While the accessibility of the end product is ultimately what matters to people with disabilities, I agree that conformance should recognise evidence that a process was followed that involved evaluation with users with disabilities and—crucially—provided data that demonstrably impacted the design in a positive way. This includes evidence that helped the design improve beyond basic technical conformance, and evidence that mitigates the impact of accessibility issues still present in the resource. 

It might even be worth giving some (minor) credit for organisations that invested in usability testing, gained some insights about how people with disabilities use the product and use technology in general, but found that they were unable to act on results due to project constraints. So long as conformance claims required some demonstration of commitment to improvement, this might help to encourage involvement of people with disabilities in the design process. But I appreciate that this is a more contentious extension of this requirement due to potential misuse. It also might be less attractive for organisations to claim.

Dave


> On Tue, Apr 9, 2019 at 9:12 AM Sahasrabudhe, Shrirang P <ssahasrabudhe@ets.org> wrote:
> I like the idea "we could consider pairing aspects of the usability testing sessions with tangible results or improvements that came directly from this testing. That way, the testing outcomes and related improvements could be linked to specific methods for instance, or techniques or what not, and we could measure just how many of the improvements came directly from involving end users with disabilities in the overall process."
> 
> We should try to go beyond the technical methods and should include some guidance on the quality processes to be followed. Not all the findings of user testing can always be incorporated due to various technical and organizational factors. Therefore, the organizations can be rewarded for documenting the findings of the user testing and the subsequent actions  they took to improve the accessibility of their product. The documentation can help the organization to clearly assess their process maturity level. It will also help adhering to the WCAG standards in a more systematic way.
> 
>  
> 
> From: Denis Boudreau [mailto:denis.boudreau@deque.com] 
> Sent: Tuesday, April 9, 2019 7:58 AM
> To: Detlev Fischer <detlev.fischer@testkreis.de>
> Cc: public-silver@w3.org
> Subject: Re: Proposal for new version of Requirement 3.7 Motivtion
> 
>  
> 
> I like the proposal with Chuck’s edits.
> 
>  
> 
> I disagree with your position Detlev, but understand your concerns. The temptation to game the system would undoubtedly rise from some of the people out there that would want to be able to claim a quick path to success (oh yeah, we tested with people, and “they” said it was fiiiiiiine...).
> 
>  
> 
> I’m just not able to agree with a statement such as:
> 
>  
> 
> “[testing]... does not in itself change the quality of the site under test. An awful site stays awful even after a lot of user testing.”
> 
>  
> 
> I believe that conducting testing with people with disabilities, when done genuinely with the goal of user experience improvements does absolutely change the quality of the site under test. The findings brought up by consulting those users is expected to bring forth positive changes. An awful site is supposed to get better as a result of the change that come from the activity of involving those users in the process. That’s just the nature of the activity. But we need a way to measure that clearly in Silver.
> 
>  
> 
> I celebrate our vision of rewarding usability testing with end users with disabilities. It does expose our model to abuse - I certainly share Detlev’s concerns here - but I’m sure that as we get to defining the details of how the scoring system will pan out, we’ll find ways to reward usability testing for aspects that actually provide value, not for things that pay lip service to the idea of making the product or service accessible. 
> 
>  
> 
> As an example, we could consider pairing aspects of the usability testing sessions with tangible results or improvements that came directly from this testing. That way, the testing outcomes and related improvements could be linked to specific methods for instance, or techniques or whatnot, and we could measure just how many of the improvements came directly from involving end users with disabilities in the overall process. The more improvements came out direct end users contributions, the higher the points.
> 
>  
> 
>  
> 
> /Denis
> 
>  
> 
> —
> 
> Denis Boudreau 
> 
> Principal accessibility SME & Training lead
> 
> Deque Systems, Inc. 
> 
> 514-730-9168
> 
>  
> 
>  
> 
>  
> 
> On Tue, Apr 9, 2019 at 04:30 Detlev Fischer <detlev.fischer@testkreis.de> wrote:
> 
> As I have said before, I think the mere fact that testing with users 
> with disabilities has taken place should not be rewarded since it does 
> not in itself change the quality of the site under test. An awful site 
> stays awful even after a lot of user testing. If then, as a result of 
> such testing, the accessibility and/or usability is improved, that 
> should impact also the conformance to measurable criteria (whether 
> absolute or score-based) - and I am happy to see those criteria extended 
> to realms so far difficult to measure.
> 
> Am 08.04.2019 um 20:42 schrieb Jeanne Spellman:
> > Here is the proposal for revision of Requirement 3.7 Motivation as 
> > requested by AGWG to make it measureable.
> >
> > Motivation
> >
> > The Guidelines motivate organizations to go beyond minimal 
> > accessibility requirements by providing a scoring system that rewards 
> > organizations that demonstrate a greater effort to improve 
> > accessibility.  For example, Methods that go beyond the minimum (such 
> > as: Methods for Guidelines that are not included in WCAG 2.x A or AA, 
> > task-completion evalations, or testing with users with disabilities) 
> > are worth more points in the scoring system.

--
David Sloan
--
UX Research Lead
The Paciello Group
https://www.paciellogroup.com

A Vispero Company
https://vispero.com
--
This message is intended to be confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, please delete this message from your system and notify us immediately.
Any disclosure, copying, distribution or action taken or omitted to be taken by an unintended recipient in reliance on this message is prohibited and may be unlawful.

Received on Tuesday, 9 April 2019 15:03:44 UTC