Re: Re[2]: acceptance criteria for new success criteria

David,
I think that we are but in slightly different ways. I agree that we want to have a "high degree of confidence that most experts would agree” but object to the idea of placing a specific, and impossible to measure, numerical metric on that.  I agree that we don’t want to say “ALL experts agree” (result of that would likely be no new SC) but ultimately the decision policy is going to dictate how we ultimately decide whether the criteria are met.

Thanks,
AWK

Andrew Kirkpatrick
Group Product Manager, Standards and Accessibility
Adobe

akirkpat@adobe.com
http://twitter.com/awkawk


From: David MacDonald <david100@sympatico.ca<mailto:david100@sympatico.ca>>
Date: Thursday, June 2, 2016 at 15:17
To: "White, Jason J" <jjwhite@ets.org<mailto:jjwhite@ets.org>>
Cc: Andrew Kirkpatrick <akirkpat@adobe.com<mailto:akirkpat@adobe.com>>, "josh@interaccess.ie<mailto:josh@interaccess.ie>" <josh@interaccess.ie<mailto:josh@interaccess.ie>>, "lisa.seeman@zoho.com<mailto:lisa.seeman@zoho.com>" <lisa.seeman@zoho.com<mailto:lisa.seeman@zoho.com>>, Patrick Lauke <redux@splintered.co.uk<mailto:redux@splintered.co.uk>>, WCAG <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Subject: Re: Re[2]: acceptance criteria for new success criteria

Hi ANdrew

PS ... are we talking about the same thing? I'm talking about that when experts are testing a web site for accessibility there should be a high level of confidence that most experts would agree.

It sounds like you are talking about when we gather together to write the standards... that is a completely separate kettle of fish... and yes I totally agree we go for consensus, not high inter reliability.

Cheers,
David MacDonald



CanAdapt Solutions Inc.

Tel:  613.235.4902

LinkedIn
<http://www.linkedin.com/in/davidmacdonald100>

twitter.com/davidmacd<http://twitter.com/davidmacd>

GitHub<https://github.com/DavidMacDonald>

www.Can-Adapt.com<http://www.can-adapt.com/>



  Adapting the web to all users

            Including those with disabilities

If you are not the intended recipient, please review our privacy policy<http://www.davidmacd.com/disclaimer.html>

On Thu, Jun 2, 2016 at 11:13 AM, White, Jason J <jjwhite@ets.org<mailto:jjwhite@ets.org>> wrote:


From: Andrew Kirkpatrick [mailto:akirkpat@adobe.com<mailto:akirkpat@adobe.com>]
Sent: Thursday, June 2, 2016 11:08 AM

David: I think the concept of high inter reliability of experts is as good as we can get.

AWK: I think that the "General agreement, characterized by the absence of sustained opposition to substantial issues” actually matches the way that the group operated in practice, and continues to do so.  What we are really doing is following the WCAG and W3C process (see http://www.w3.org/WAI/GL/decision-policy), and our goal is unanimity, or an absence of objections.  Whether this means that everyone actively agrees or if most people agree and the rest can live with it doesn’t really matter.

I’m concerned about the “8 of 10” since it starts to feel like voting and that hasn’t been the way that we have operated historically.

I think you’re conflating (1) how the working group reaches consensus, which as you describe, and (2) the standard of inter-rater reliability that the working group considers success criteria need to have in order to be included. The latter has nothing to do with voting – see my formulation of it earlier in this thread.


________________________________

This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.


Thank you for your compliance.

________________________________