Re: This is significantly different from what was agreed. - was Re: CFC: Manual testing processes

This was discussed on a call, and the people on that call reached a 
resolution [1]. This is the resolution that was recorded in the minutes 
of that meeting:

"RESOLUTION: User testing is not a required part of a manual testing 
process for WCAG test criteria."

A Call For Consensus (CFC) is now open. This is to give the whole 
Working Group (WG) the chance to vote on it.

For those people not familiar with this W3C process, it is where every 
member of a WG casts a vote on a resolution. WG participants can do one 
of the following things:

- Reply to the CFC email to show support for the resolution
- Reply to the CFC email to show opposition to the resolution
- Do nothing (which is taken as silent support for the resolution)

So it seems to me that the resolution is clear, and that now the WG is 
voting whether to accept it or not.

Léonie
[1] https://www.w3.org/2017/02/07-ag-minutes.html
-- 
@LeonieWatson tink.uk Carpe diem

On 15/02/2017 15:39,  .seeman wrote:
> As is your choice.
>
> what I am saying is that the resolution passed should not now include
>  this without a full debate and new vote.
>
> All the best
>
> Lisa Seeman
>
> LinkedIn <http://il.linkedin.com/in/lisaseeman/>, Twitter
> <https://twitter.com/SeemanLisa>
>
>
>
>
> ---- On Wed, 15 Feb 2017 17:32:26 +0200 *John
> Foliot<john.foliot@deque.com>* wrote ----
>
>     TL;DR:
>
>     I will strongly oppose any Success Criteria that insists on
>     user/usability testing (even for exemptions), as the very nature of
>     that testing is subjective, and thus non-repeatable at scale.
>
>     *****
>
>     > For example use active voicing unless user testing with five
>     people with cognitive disabilities has shown passive voicing to be
>     clearer.
>
>     Hi Lisa,
>
>     I now note that you have attached a quantity - *_5_* People with
>     Cognitive Disabilities - to this exception that is not in the Draft
>     SC. Is that a proposed edit to the emergent Success Criteria?
>
>     While I can understand this desire, the problem here is that it is
>     next to impossible to turn this into a measurable metric. Two teams
>     of 5 testers could emerge with 2 different 'responses' from
>     user-testing (one thumbs-up, the other thumbs-down) and it would be
>     impossible to verify who would be correct.
>
>     I also remain concerned over the definition of "clearer", as again
>     this is not a measurable metric.
>
>     Additionally, for compliance and monitoring tracking, how could this
>     be recorded? Will corporate entities be required to maintain those
>     user-testing logs in case they are challenged in court? That might
>     sound extreme, it is however also a very legitimate scenario (and
>     question), and proceeding with this clause in the SC will, I
>     suspect, be the Achilles heal and will ultimately result in this SC
>     being rejected.
>
>     Requirements need to be repeatable, testable, and achievable on both
>     small boutique sites, as well as the largest multi-national,
>     multi-language sites. Demanding of large corporate clients that
>     *all* corporate editorial content must be reviewed by 5 disabled
>     users before claiming conformance (or an exception/exemption) will
>     meet significant resistance, up-to and including non-conformance
>     (see my earlier note about 'Undue Burden').
>
>     JF
>
>
>
>
>     On Wed, Feb 15, 2017 at 8:07 AM, lisa.seeman <lisa.seeman@zoho.com
>     <mailto:lisa.seeman@zoho.com>> wrote:
>
>         __
>         This is hard to explain by email.,
>
>         We agreed that we were not going to make disability testing a
>         blanket requirement for conformance.
>         However in some cases that is not the same as what Andrew is
>         saying -  "that if the only way to test a success criteria is to
>         conduct user testing, then that is not “testable” with regard to
>         WCAGT 2.1. "
>
>
>         For example, we have a SC that requires you to use active
>         voicing on critical information - but we have an exception:
>
>         *Exceptions:*
>
>           * When a passive voice or a tense (other than present tense)
>             is clearer. Other voices or tenses may be used when it has
>             been shown, via user testing, to be easier to understand,
>             friendlier, or appropriate.
>
>         (In fact we have many exceptions but this is the one that is
>         relevant here)
>
>         Here we allow user testing to activate an exception.
>
>         Is that still OK with the implication that Andrew has mentioned
>         for this resolution? This is unclear to me. If it is it increase
>         the scope of what was agreed.
>
>
>         All the best
>
>         Lisa Seeman
>
>         LinkedIn <http://il.linkedin.com/in/lisaseeman/>, Twitter
>         <https://twitter.com/SeemanLisa>
>
>
>
>
>         ---- On Wed, 15 Feb 2017 15:22:51 +0200 *Joshue O
>         Connor<josh@interaccess.ie <mailto:josh@interaccess.ie>>* wrote
>         ----
>
>             >The resolution implication is different to what was
>             discussed. We CAN NOT pass the >resolution if this
>             implication does not allow for exceptions via user testing
>             at least >without a real discussion so we all understand
>             what is at stake
>
>             Sorry Lisa. I don't understand your point here. If anyone
>             does user testing and finds issues for any given target
>             group then they are totally free to flag those issues and
>             make recommendations on the outputs or results of a test.
>
>             There is no explicit restriction here that I can see, can
>             you clarify what you mean?
>
>             Thanks
>
>             Josh
>
>                 lisa.seeman <mailto:lisa.seeman@zoho.com>
>                 15 February 2017 at 13:15
>                 The resolution implication is different to what was
>                 discussed. We CAN NOT pass the resolution if this
>                 implication does not allow for exceptions via user
>                 testing at least without a real discussion so we all
>                 understand what is at stake
>
>                 We agreed we were not making user testing a requirement
>                 for conformance.
>
>                 This implication is significantly different and changes
>                 things.
>
>                 User testing was ok to enable an exception. In other
>                 words it is not required, but you can claim an exception
>                 via use testing.
>                 For example use active voicing unless user testing with
>                 five people with cognitive disabilities has shown
>                 passive voicing to be clearer.
>
>                 This implication has not been discussed . The vote is
>                 meaningless if this "implication" has nt been fully
>                 understood by everyone voting
>
>                 This add be shown to be
>
>                 All the best
>
>                 Lisa Seeman
>
>                 LinkedIn <http://il.linkedin.com/in/lisaseeman/>,
>                 Twitter <https://twitter.com/SeemanLisa>
>
>
>
>
>                 ---- On Wed, 15 Feb 2017 08:59:19 +0200
>                 *Chakravarthula<srchakravarthula@informatica.com>
>                 <mailto:srchakravarthula@informatica.com>* wrote ----
>
>
>                 Chakravarthula, Srinivasu
>                 <mailto:srchakravarthula@informatica.com>
>                 15 February 2017 at 06:59
>                 +1
>
>                 Regards,
>                 Srinivasu Chakravarthula | Informatica | @CSrinivasu
>                 Sent from my iPhone
>
>                 Andrew Kirkpatrick <mailto:akirkpat@adobe.com>
>                 14 February 2017 at 03:19
>                 Call For Consensus — ends Wednesday February 15th at
>                 10:30pm Boston time.
>
>                 The requirements for WCAG 2.1 SC's
>                 (https://www.w3.org/WAI/GL/wiki/WCAG_2.1_Success_Criteria have
>                 been discussed at length. Included in the requirements
>                 is #2, which states "Be testable through automated or
>                 manual processes”, which indicates that in order for a
>                 success criteria to be regarded as “testable” it must be
>                 possible to determine whether a page passes that
>                 criteria using automated or manual testing processes.
>
>                 On last Tuesday’s call the WG came to a resolution
>                 regarding this item, specifically related to user
>                 testing. The group also surveyed this question, and
>                 arrived at a unanimous agreement:
>
>                 "User testing is not a required part of a manual testing
>                 process for WCAG test criteria.”
>
>                 This resolution indicates that if the only way to test a
>                 success criteria is to conduct user testing, then that
>                 is not “testable” with regard to WCAGT 2.1.
>
>                 The Working Group will recommend strongly in WCAG 2.1
>                 (as it did in WCAG 2.0) that user testing be conducted.
>
>                 For background:
>                 Call minutes: http://www.w3.org/2017/02/07-ag-minutes.html
>                 Survey: https://www.w3.org/2002/09/wbs/35422/testing20170207/results#xq1
>
>                 If you have concerns about this proposed consensus
>                 position that have not been discussed already and feel
>                 that those concerns result in you “not being able to
>                 live with” this decision, please let the group know
>                 before the CfC deadline.
>
>                 Thanks,
>                 AWK
>
>                 Andrew Kirkpatrick
>                 Group Product Manager, Standards and Accessibility
>                 Adobe
>
>                 akirkpat@adobe.com <mailto:akirkpat@adobe.com>
>                 http://twitter.com/awkawk
>
>
>             --
>             Joshue O Connor
>             Director | InterAccess.ie
>
>
>
>
>
>
>     --
>     John Foliot
>     Principal Accessibility Strategist
>     Deque Systems Inc.
>     <mailto:john.foliot@deque.com>john.foliot@deque.com
>     <mailto:john.foliot@deque.com>
>
>     Advancing the mission of digital accessibility and inclusion
>
>
>

Received on Wednesday, 15 February 2017 16:00:00 UTC