Re: Measurable vs testable [was: Silver Requirements Issues]

​> and isn't a document that I remember seeing before

It's my own tracking tool that I've shared with the group in case its
helpful.​

Cheers,
David MacDonald



*Can**Adapt* *Solutions Inc.*

Tel:  613.235.4902

LinkedIn
<http://www.linkedin.com/in/davidmacdonald100>

twitter.com/davidmacd

GitHub <https://github.com/DavidMacDonald>

www.Can-Adapt.com <http://www.can-adapt.com/>



*  Adapting the web to all users*
*            Including those with disabilities*

If you are not the intended recipient, please review our privacy policy
<http://www.davidmacd.com/disclaimer.html>

On Mon, Jul 16, 2018 at 12:58 PM, Jeanne Spellman <
jspellman@spellmanconsulting.com> wrote:

> Thanks, David!  This document is very helpful, and isn't a document that I
> remember seeing before.  I have been working off an AGWG wiki page and the
> Silver designation of the 2.1 Gitgub Issues, but this is a much more useful
> form factor.  Bookmarked.  :)
>
> We are NOT writing content yet, and I am relying on the various task
> forces to recommend 1 or 2 examples that they think are representative for
> Silver prototypes and user testing.  However, the notes are very helpful in
> identifying examples that would be useful for Silver the TFs may not have
> considered.
>
> Thank you very much!
>
> jeanne
>
> On 7/16/2018 12:47 PM, David MacDonald wrote:
>
> Here are a list of SCs that didn't make it into WCAG.
>
> http://tinyurl.com/jmo9st4
>
> They are found on the tab
> "SCs Not accepted for 2.1"
>
> Cheers,
> David MacDonald
>
>
>
> *Can**Adapt* *Solutions Inc.*
>
> Tel:  613.235.4902
>
> LinkedIn
> <http://www.linkedin.com/in/davidmacdonald100>
>
> twitter.com/davidmacd
>
> GitHub <https://github.com/DavidMacDonald>
>
> www.Can-Adapt.com <http://www.can-adapt.com/>
>
>
>
> *  Adapting the web to all users*
> *            Including those with disabilities*
>
> If you are not the intended recipient, please review our privacy policy
> <http://www.davidmacd.com/disclaimer.html>
>
> On Thu, Jul 12, 2018 at 9:49 AM, Rochford, John <
> john.rochford@umassmed.edu> wrote:
>
>> Hi Jeanne,
>>
>> I am the COGA Task Force manager for the Accessible Authentication SC. I
>> am also the author of the Web Security and Privacy issue paper it is based
>> on. See: https://rawgit.com/
>> <https://rawgit.com/w3c/coga/master/issue-papers/privacy-security.html>
>> w3c
>> <https://rawgit.com/w3c/coga/master/issue-papers/privacy-security.html>/
>> <https://rawgit.com/w3c/coga/master/issue-papers/privacy-security.html>
>> coga
>> <https://rawgit.com/w3c/coga/master/issue-papers/privacy-security.html>
>> /master/issue-papers/privacy-security.html
>> <https://rawgit.com/w3c/coga/master/issue-papers/privacy-security.html>
>>
>> Accessible Authentication was the only COGA SC considered at Level A for
>> 2.1. That it did not make it was due in part to the testability issues you
>> and Alastair are discussing. However, the push back involved several other
>> objections.
>>
>> If you would like, I would be pleased to be involved in the effort to
>> develop the Accessible Authentication SC.
>>
>>
>> John
>>
>> John Rochford
>> UMass Medical School
>> Eunice Kennedy Shriver Center
>> Director, INDEX Program
>> Faculty, Family Medicine & Community Health
>> www.DisabilityInfo.org <http://%3Cbr/%3Ewww.DisabilityInfo.org>
>>
>> ------------------------------
>> *From:* Alastair Campbell <acampbell@nomensa.com>
>> *Sent:* Thursday, July 12, 2018 8:21:35 AM
>> *To:* Jeanne Spellman; w3c-wai-gl@w3.org
>> *Subject:* Re: Measurable vs testable [was: Silver Requirements Issues]
>>
>>
>> Hi Jeanne,
>>
>>
>>
>> A couple of things I’d like to contribute/clarify:
>>
>>
>>
>> > JS: The test example we have been discussing is the COGA proposal for
>> accessible authentication that was not included in 2.1.
>>
>>
>>
>> The main issues there were more around feasibility rather than
>> testability, the more difficult ones were things like Plain Language in
>> navigation, as the context of the site has a big impact, and testability is
>> less clear than most interface level things. Usability testing wouldn’t be
>> the right tool for the job there, you’d need huge sample sizes.
>>
>>
>>
>>
>>
>> > For example, Silver could define a usability test procedure -- based on
>> standard UX techniques -- that any developer, QA tester, or a11y expert
>> could perform themselves (they would not have to have a outside PwD test
>> it) to determine if a login page conformed to Silver for accessible
>> authentication.
>>
>> I am very sceptical that this approach would provide a good ‘answer’ to
>> the question. Even apart from the ease with which testing can be biased
>> (and in this case there is good motivation to bias the results
>> intentionally!), what would constitute a pass? 3 out of 6 people? It’s
>> never 100%. There are fundamental issues with using usability testing in a
>> direct way.
>>
>>
>>
>> Also, what if there is a pattern out there which has been usability
>> tested by others already, with published results, do you still need to test
>> yours?
>>
>>
>>
>> > While we have discussed using process as a means of measurement, that
>> has not been our focus and probably would only apply to the higher levels
>> of Silver -- if we include it at all.  There are issues with using process
>> to determine accessibility and we don't have solutions for them yet.  We
>> are eager to get ideas from the people with experience with using process
>> to evaluate accessibility.
>>
>> That’s fair (I’d say the same about usability testing), but I’d like to
>> include it as an avenue to explore in the requirements.
>>
>> I was going to create a PR but I don’t have write access on the repo
>> (yet).
>>
>> My suggested additions were:
>>
>> <h4>Conformance Model</h4>
>>
>> + <p>There are several areas for exploration in how conformance can work.
>> These opportunities may or may not be incorporated. Then need to work
>> together, and that interplay will be governed by the design principles.</p>
>>
>> Add a bullet under flexibility:
>>
>> + <li><strong>Process guidance</strong>: Some requirements may be more
>> important for certain websites and be very dependent on context. For
>> example using plain language terminology in navigation is very difficult to
>> test reliably given the constraints and context an ecommerce site has
>> compared to a public sector website. Providing for a process to follow and
>> document may be more appropriate for some requirements than either
>> measurable or task-based approaches.</li>
>>
>> And on tech-neutral:
>>
>> <h3>Technology Neutral</h3>
>>
>> + <p>The guidelines should cover all web technologies available to users.
>> It is likely that a layer of the guidance will be written to be technology
>> neutral, but the guidelines should be able to include criteria that do not
>> work across all technologies.</p>
>> <p class="ednote">Technology neutral
>>
>>
>>
>> Cheers,
>>
>> -Alastair
>>
>>
>>
>
>
>