Re[2]: acceptance criteria for new success criteria

I think it sounds good, seems reasonable etc but as Patrick points out - 
not really practically testable in and of itself. It may also lend 
itself to the sway of the court of public opinion or received wisdom 
(albeit esteemed and informed) or worse  to 'group think' and/or lazy 
assumptions about what is best in a given situation.

Who will willingly play Cassandra in a court of her peers? [1]

[1] https://en.wikipedia.org/wiki/Cassandra

------ Original Message ------
From: "lisa.seeman" <lisa.seeman@zoho.com>
To: "White" <jjwhite@ets.org>
Cc: "Patrick H. Lauke" <redux@splintered.co.uk>; "w3c-wai-gl@w3.org" 
<w3c-wai-gl@w3.org>
Sent: 02/06/2016 13:52:31
Subject: RE: acceptance criteria for new success criteria

>I am happy with this standard
>
>
>"
>In practice, the standard was: "participants in the working group, by 
>consensus, are confident that 8 out of 10 informed evaluators would 
>agree in their application of the proposed success criterion" (across a 
>wide range of cases, I assume, though this last point is usually left 
>implicit). "
>
>
>All the best
>
>Lisa Seeman
>
>LinkedIn, Twitter
>
>
>
>
>---- On Thu, 02 Jun 2016 15:43:59 +0300 White<jjwhite@ets.org> wrote 
>----
>>
>>
>> > -----Original Message-----
>> > From: Patrick H. Lauke [mailto:redux@splintered.co.uk]
>> > Sent: Thursday, June 2, 2016 4:57 AM
>>
>>
>> > It may be a minor point, but: I'd prefer language that's a little 
>>less...specific.
>> > Giving an actual figure of "8 out of 10" gives it a whiff of "it can 
>>be proven with
>> > hard numbers", sure, but really: if there's ever a disagreement, do 
>>we really
>> > expect somebody to gather 10 experts, get their opinions, and then 
>>make go for
>> > the option that had 8 votes? What if it's 5 out of 10...a draw 
>>(which is probably
>> > why you'd want 9 experts to be able to determine at least majority, 
>>barring
>> > abstentions).
>>
>>In practice, the standard was: "participants in the working group, by 
>>consensus, are confident that 8 out of 10 informed evaluators would 
>>agree in their application of the proposed success criterion" (across 
>>a wide range of cases, I assume, though this last point is usually 
>>left implicit).
>>
>>So far as I am aware, no one has empirically tested the extent to 
>>which this standard is met by the success criteria that ultimately 
>>comprised WCAG 2.0.
>>
>>
>>________________________________
>>
>>This e-mail and any files transmitted with it may contain privileged 
>>or confidential information. It is solely for use by the individual 
>>for whom it is intended, even if addressed incorrectly. If you 
>>received this e-mail in error, please notify the sender; do not 
>>disclose, copy, distribute, or take any action in reliance on the 
>>contents of this information; and delete it from your system. Any 
>>other use of this e-mail is prohibited.
>>
>>
>>Thank you for your compliance.
>>
>>________________________________
>
>