RE: Acceptance Criteria for proposals for new Success Criteria

From: Andrew Kirkpatrick [mailto:akirkpat@adobe.com]
Sent: Wednesday, August 3, 2016 4:12 PM

Ah, I see.  I don’t think that will work.  Testability is an absolute requirement.  If an SC isn’t testable then it won’t become an SC, as it was with WCAG 2.0.  Worth noting that there is a difference between “testable” and “machine testable” and this covers both.
[Jason] Being reliably testable is not a straightforward property that an SC either possesses or lacks. This is especially true of human-testable requirements, which can be made more or less reliably testable (i.e., with more or less probability of agreement among informed reviewers).
There’s an interesting paper which provides evidence and analysis to suggest that we were not as effective as we hoped we would be in defining reliably human testable success criteria – see the reference below. In particular, in a study of a sample of Web pages, 80% agreement even among informed reviewers with accessibility expertise was not achieved; yet this was once the degree of agreement that we sought to reach in defining reliably human testable SCs.
I think we should strive for more reliable testing, but not at the expense of providing general and useful requirements that solve accessibility problems. There is also a need for more research into ways of improving the extent to which success criteria are reliably testable (especially in the next generation of WAI Guidelines).

The reference is:

Brajnik, G., Yesilada, Y., & Harper, S. (2012). Is accessibility conformance an elusive property? A study of validity and reliability of WCAG 2.0. ACM Transactions on Accessible Computing (TACCESS), 4(2), 8.

6.Apply to all content, unless specific exceptions are included in the success criteria (e.g. "except interruptions involving an emergency").
not sure what I’m missing.  The original text says that if there are exceptions that they needs to be included in the SC.  That is explicitly mentioning them, isn’t it?
[Jason] My point is that exceptions and preconditions often aren’t the same. Stating that a requirement applies in some well defined circumstances (but not in most cases) isn’t an example of providing an exception to a general rule. “Preconditions for the application of success criteria should be stated explicitly” is the requirement, I think.

________________________________

This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.


Thank you for your compliance.

________________________________