Re[2]: acceptance criteria for new success criteria

These are good questions. I think we need to (somehow) try to evolve 
this beyond '8 out of 10 cats prefer'.

Thanks

Josh

------ Original Message ------
From: "White, Jason J" <jjwhite@ets.org>
To: "Patrick H. Lauke" <redux@splintered.co.uk>; "w3c-wai-gl@w3.org" 
<w3c-wai-gl@w3.org>
Sent: 02/06/2016 13:43:59
Subject: RE: acceptance criteria for new success criteria

>
>
>>  -----Original Message-----
>>  From: Patrick H. Lauke [mailto:redux@splintered.co.uk]
>>  Sent: Thursday, June 2, 2016 4:57 AM
>
>
>>  It may be a minor point, but: I'd prefer language that's a little 
>>less...specific.
>>  Giving an actual figure of "8 out of 10" gives it a whiff of "it can 
>>be proven with
>>  hard numbers", sure, but really: if there's ever a disagreement, do 
>>we really
>>  expect somebody to gather 10 experts, get their opinions, and then 
>>make go for
>>  the option that had 8 votes? What if it's 5 out of 10...a draw (which 
>>is probably
>>  why you'd want 9 experts to be able to determine at least majority, 
>>barring
>>  abstentions).
>
>In practice, the standard was: "participants in the working group, by 
>consensus, are confident that 8 out of 10 informed evaluators would 
>agree in their application of the proposed success criterion" (across a 
>wide range of cases, I assume, though this last point is usually left 
>implicit).
>
>So far as I am aware, no one has empirically tested the extent to which 
>this standard is met by the success criteria that ultimately comprised 
>WCAG 2.0.
>
>
>________________________________
>
>This e-mail and any files transmitted with it may contain privileged or 
>confidential information. It is solely for use by the individual for 
>whom it is intended, even if addressed incorrectly. If you received 
>this e-mail in error, please notify the sender; do not disclose, copy, 
>distribute, or take any action in reliance on the contents of this 
>information; and delete it from your system. Any other use of this 
>e-mail is prohibited.
>
>
>Thank you for your compliance.
>
>________________________________

Received on Thursday, 2 June 2016 13:27:59 UTC