Re: This is significantly different from what was agreed. - was Re: CFC: Manual testing processes

As is your choice.

what I am saying is that the resolution passed should not now include  this without a full debate and new vote.

All the best

Lisa Seeman

LinkedIn, Twitter





---- On Wed, 15 Feb 2017 17:32:26 +0200 John Foliot<john.foliot@deque.com> wrote ---- 

TL;DR:

I will strongly oppose any Success Criteria that insists on user/usability testing (even for exemptions), as the very nature of that testing is subjective, and thus non-repeatable at scale.



*****


> For example use active voicing unless user testing with five people with cognitive disabilities has shown passive voicing to be clearer. 


Hi Lisa,


I now note that you have attached a quantity - 5 People with Cognitive Disabilities - to this exception that is not in the Draft SC. Is that a proposed edit to the emergent Success Criteria?


While I can understand this desire, the problem here is that it is next to impossible to turn this into a measurable metric. Two teams of 5 testers could emerge with 2 different 'responses' from user-testing (one thumbs-up, the other thumbs-down) and it would be impossible to verify who would be correct. 


I also remain concerned over the definition of "clearer", as again this is not a measurable metric.


Additionally, for compliance and monitoring tracking, how could this be recorded? Will corporate entities be required to maintain those user-testing logs in case they are challenged in court? That might sound extreme, it is however also a very legitimate scenario (and question), and proceeding with this clause in the SC will, I suspect, be the Achilles heal and will ultimately result in this SC being rejected.


Requirements need to be repeatable, testable, and achievable on both small boutique sites, as well as the largest multi-national, multi-language sites. Demanding of large corporate clients that *all* corporate editorial content must be reviewed by 5 disabled users before claiming conformance (or an exception/exemption) will meet significant resistance, up-to and including non-conformance (see my earlier note about 'Undue Burden'). 


JF








On Wed, Feb 15, 2017 at 8:07 AM, lisa.seeman <lisa.seeman@zoho.com> wrote:
This is hard to explain by email.,


We agreed that we were not going to make disability testing a blanket requirement for conformance. 
However in some cases that is not the same as what Andrew is saying -  "that if the only way to test a success criteria is to conduct user testing, then that is not “testable” with regard to WCAGT 2.1. "




For example, we have a SC that requires you to use active voicing on critical information - but we have an exception:

Exceptions:
When a passive voice or a tense (other than present tense) is clearer. Other voices or tenses may be used when it has been shown, via user testing, to be easier to understand, friendlier, or appropriate.
(In fact we have many exceptions but this is the one that is relevant here)


Here we allow user testing to activate an exception. 


Is that still OK with the implication that Andrew has mentioned for this resolution? This is unclear to me. If it is it increase the scope of what was agreed.



All the best

Lisa Seeman

LinkedIn, Twitter





---- On Wed, 15 Feb 2017 15:22:51 +0200 Joshue O Connor<josh@interaccess.ie> wrote ---- 

 >The resolution implication is different to what was discussed. We CAN NOT pass the >resolution if this implication does not allow for exceptions via user testing at least >without a real discussion so we all understand what is at stake

Sorry Lisa. I don't understand your point here. If anyone does user testing and finds issues for any given target group then they are totally free to flag those issues and make recommendations on the outputs or results of a test. 

There is no explicit restriction here that I can see, can you clarify what you mean?

Thanks

Josh
 
  lisa.seeman
  15 February 2017 at 13:15
 

The resolution implication is different to what was discussed. We CAN NOT pass the resolution if this implication does not allow for exceptions via user testing at least without a real discussion so we all understand what is at stake
We agreed we were not making user testing a requirement for conformance.

This implication is significantly different and changes things. 


User testing was ok to enable an exception. In other words it is not required, but you can claim an exception via use testing.
For example use active voicing unless user testing with five people with cognitive disabilities has shown passive voicing to be clearer. 


This implication has not been discussed . The vote is meaningless if this "implication" has nt been fully understood by everyone voting


This add be shown to be 

All the best

Lisa Seeman

LinkedIn, Twitter





---- On Wed, 15 Feb 2017 08:59:19 +0200  Chakravarthula<srchakravarthula@informatica.com> wrote ---- 










  Chakravarthula, Srinivasu
  15 February 2017 at 06:59
 

+1

Regards,
Srinivasu Chakravarthula | Informatica | @CSrinivasu
Sent from my iPhone



  Andrew Kirkpatrick
  14 February 2017 at 03:19
 

 Call For Consensus — ends Wednesday February 15th at 10:30pm Boston time.
  
 
  The requirements for WCAG 2.1 SC's (https://www.w3.org/WAI/GL/wiki/WCAG_2.1_Success_Criteria have been discussed at length. Included in the requirements is #2, which states "Be testable through automated or manual processes”, which indicates that in order for a success criteria to be regarded as “testable” it must be possible to determine whether a page passes that criteria using automated or manual testing processes.
  
 
  On last Tuesday’s call the WG came to a resolution regarding this item, specifically related to user testing. The group also surveyed this question, and arrived at a unanimous agreement:
  
 
  "User testing is not a required part of a manual testing process for WCAG test criteria.”
  
 
  This resolution indicates that if the only way to test a success criteria is to conduct user testing, then that is not “testable” with regard to WCAGT 2.1.
  
 
  The Working Group will recommend strongly in WCAG 2.1 (as it did in WCAG 2.0) that user testing be conducted.
  
 
  For background:
  Call minutes: http://www.w3.org/2017/02/07-ag-minutes.html
  Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results#xq1
  
 
 If you have concerns about this proposed consensus position that have not been discussed already and feel that those concerns result in you “not being able to live with” this decision, please let the group know before the CfC deadline.
  
 
   Thanks,
 AWK
 
 
 Andrew Kirkpatrick
 Group Product Manager, Standards and Accessibility
 Adobe 
 
 
 akirkpat@adobe.com
 http://twitter.com/awkawk
 
 
 

-- 
Joshue O Connor
 Director | InterAccess.ie 
 

 













-- 
John Foliot


Principal Accessibility Strategist

Deque Systems Inc.

john.foliot@deque.com



Advancing the mission of digital accessibility and inclusion