RE: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Hi all,

At the request of the COGA team, let’s avoid using “Making Content Usable” as an example of a protocol. It could imply that the information contained in there won’t be used in methods and other parts of WCAG 3.0, which isn’t necessarily the case. Plain language is a good example, especially for clear language when applied to content in English (maybe only in North America, not sure if it applies globally to all English speaking regions of the world).

I am a little concerned that it’ll be difficult to make progress if we focus on a definition without giving concrete examples about how protocols will be used. It can be used to measure inputs and not outcomes in any of those scenarios identified in the agenda and it’s hard to picture what would make a good protocol without understanding how it’ll be used exactly.

For user testing, I would imagine that companies of different sizes would implement protocols that would apply to companies of their size and would make sense for their business. User testing can be approached in theory in similar ways, however, no matter an organization’s size – you’ll still need to recruit a panel, define scope, etc. An organization, could then, in theory, use a protocol to help them create that process. It could also define a process for adding insights gained from user testing into an organization’s backlog or define ways an organization can implement/categorize feedback.

With the user process example (like with screen reader testing), you would likely have an organization adopt guidance about how to perform screen reader testing generally that would include how screen readers work, how to test different kinds of functionality and such and then some expected behavior. This would be general guidance that can be applied to multiple guidelines and methods and would represent some general best practices that can be applied at scale. It’s likely that different kinds of folks using screen reader software for testing will get wildly different results (as a person who uses screen readers every day would have a different perspective than a QA tester), but the protocol can put some structure around that to help organizations achieve something more consistent. 😊

Is there a single particular use case that everyone is moving towards? If so, we might want to define what that is and that might help us move forward. If it’s only for agenda item 1, for example and that’s the consensus, then we might want to define which (exactly) standards are so subjective that a protocol would help an organization achieve an outcome:


  1.  Would it only be clear language?
  2.  If there are other standards that would apply, would any of them come from WCAG 2.x or only in some of the newer outcomes being defined in WCAG 3.0?
  3.  In either case listed in two, would it be helpful to create a list?

Seeing the universe of cases where protocols may apply could be helpful for defining the requirements for an acceptable protocol, but that definitely could just be my own opinion.

What do folks think?

Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://www.accessibilityassociation.org/s/wascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.]  [Digital A11ies -- Working Together for All]

From: John Foliot <john@foliot.ca>
Sent: Wednesday, April 6, 2022 7:40 PM
To: Jennifer Strickland <jstrickland@mitre.org>
Cc: Jaunita George <jaunita_george@navyfederal.org>; public-silver@w3.org; w3c-wai-gl@w3.org; Benjamin Feigel <Benjamin_Feigel@navyfederal.org>
Subject: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Hi All

A strong +1 to Jenn's comment, and a reminder that we had originally agreed that "Protocols measure inputs and not outcomes." (7 January 2022<https://secure-web.cisco.com/1DnoYE9Nk9im8xHHdlEwNUiP3J3o9S1NJgh3iKPH60aqH7Vjbpo5op7zW0HCvTTxGcJj0LFeqnZcHmTu3YQrIOxsRl7XIGISptTBogKMGL6AyIiamnm0rgm6IwTI2zwecIi7mYNxVFInR2lcuj9jY1lVhdVnaZn3IiRzaDPoMLhLkW1_F1K9PoWXm5PtCk_N8HWwVwPHtNOqkBXNsz88651maxViYSm0z7eRmsy2U963vxFGU2nSDJGFDrdSNUZv2cRhCMc2VWC-zjb7qwzJY6SlpuVLuy4olwD0TRVOC4L7qkzX4CJmXRGP9H1l1Aatuc3XHajIjYOGANt1dfd2pa4fmYBtQzo9SEWMXj9WDtnCdkOkXpSkZecsY7FU4W8Icc6h1hNCH9unJp1ayFtJ1RKS7zg87-SQzeDa0XEtAHuk6YhzCDUoPKeLRdQIBIST1ZW8EMPMfo1tmqfQvxKJMfA/https%3A%2F%2Fgithub.com%2Fw3c%2Fsilver%2Fwiki%2FProtocols%23key-decisions-agreed-on-with-date>).

Yet, as I read the agenda, it still feels like you want to look at outcomes. But this is where I think we get stuck - because determining outcomes for things that cannot be measured using ACT-Rules-like requirements will remain subjective and is the real problem.

Why a problem?
Because if/when you ask an entity whether they succeeded or not, *of course* they will claim that *in their opinion* they have. Yet, in the context of legislation, *of course* the litigant will say the contrary, that they haven't. You're still trying to measure outcomes!

Attempting to measure these points of view cannot (I assert) be measured, for the basic reason that they are opinions. (...and as I used to tease my daughter, "everyone is entitled to a wrong opinion" ;-) )

Continuing:

  *   Protocol for how to perform testing against a user process. (JF: useful, but not in the context of actually making the content more accessible, a 'protocol' like this would outline the steps you need to take when testing, but does nothing to guide or inform content creators. And how would a user-process be scoped, by whom, and how/why? Every time you introduce a potential fork in the user-path [clicking on a help icon for example] you have to build out your 'flow' to account for that... it doesn't scale! And "happy path" testing will usually 'pass', by my experience it's when the user has to deviate from the happy path that things start to go sideways...)
  *   Protocol for how an organization can do user testing (JF: Again, a useful set of guidance, but it may not scale either: Susan's Flower Shoppe (with 3 stores in the tri-state area) will simply not be in a position to do the same type of user-testing as Amazon or Facebook, and attempting to determine any kind of stratification (different processes for different sized orgs) will also introduce a real quagmire (where do you draw the lines, and why?). I think the best you could ever get there would be an assertion that user-testing was performed on [date] for the following flows [list flows] - but... does that testing then absolutely ensure more accessibility? (NOPE) The real win is taking results from user-testing and applying that knowledge in the next round of development. The real value of user testing is what you learn from the testing, and not the actual testing itself.)
Might I then respectfully suggest that rather than kicking off asking "how we can use a protocol" that instead we seek consensus on "what makes a candidate protocol acceptable for use in WCAG 3".

If we remain true to earlier agreements (Protocols measure inputs and not outcomes) then I will suggest that a key commonality would be that it represents outcomes and guidance geared towards the creation process, and NOT the testing/evaluation/measuring process. Shift Left in practice!

I personally envision adopting protocols as essentially promising (publicly - for the accountability piece) to do the requisite research to achieve the outcomes as described, and I argue that winning that education battle is worthwhile in-and-of-itself.

So when Making Content...COGA or PlainLanguage.gov outline Outcomes and then explain the issue and strategies that individual entities could apply *in context* to their content, they are in fact 'teaching' - and I assert THAT is the real value of Protocols (as I envision the definition of the term related to WCAG 3).

So with that definition, now Susan's Flower Shoppe and Amazon could both "learn what makes Plain Language" and then apply that learning to their content IN CONTEXT - scale is no longer a problem in the traditional sense (although it will be harder for larger orgs to remain consistent - but they will also be in a better position to have policies and processes in place due to the size of their org)

JF
(who hopes he can join the call Friday morning... stand by)





On Wed, Apr 6, 2022 at 4:05 PM Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>> wrote:
Hi Jaunita and all,
Thanks for communicating an agenda ahead of time. I think this can be very helpful and avoid using the meeting time to agree.
Regarding #2, “Selecting and writing one protocol from scratch as an exercise,” I don’t think we would write a protocol. We could document how one might document using a protocol.
Previously we said we agreed to use PlainLanguage.gov as the protocol and then ended up evaluating what the US Department of Labor documented for their efforts to meet, as I read it, the Plain Writing Act, which is related but a law rather than a protocol. Now there’s a proposal to test “Protocol for how to perform testing against a user process,” using screen reader testing as a user process — but do we have a protocol to use?
Can we agree upon a protocol and site to test, go through the process of how a person might do that, as we previously agreed to do?
Thanks,
Jennifer

From: Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>>
Date: Wednesday, April 6, 2022 at 3:38 PM
To: "public-silver@w3.org<mailto:public-silver@w3.org>" <public-silver@w3.org<mailto:public-silver@w3.org>>, "w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>" <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Cc: Benjamin Feigel <Benjamin_Feigel@navyfederal.org<mailto:Benjamin_Feigel@navyfederal.org>>
Subject: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan
Resent-From: <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Resent-Date: Wednesday, April 6, 2022 at 3:37 PM

Hi All,

At the last meeting the team mentioned that I’ll be stepping in as a co-lead for the Protocols sub-group. I’m really excited to be working with you all in this capacity and to do what I can to further this discussion. I’m new to this, so please don’t hesitate to reach out if I make a mistake or forget something – I’m always available by email and am happy to also set aside time to meet and discuss any questions or comments you have. I really look forward to talking to you all on Friday. 😊

New meeting time:

Last meeting, we agreed to move our regular Friday meeting to 8:00am instead of 9:00am EST. This means that the Protocols Subgroup will be meet this Friday, April 8th at 8:00 AM Boston Time (1400 UTC). I sent out an invite, but please let me know if you didn’t receive it.

Plan for the next few meetings:

To help us answer some of these excellent questions we’ve tackling, I thought we could focus our efforts in the next few meetings to:


  1.  Achieving consensus on how we can use a protocol.



After we’ve achieved consensus on that question, we can move on to:

  1.  Selecting and writing one protocol from scratch as an exercise.

This may help us structure our discussions and help us continue moving forward on all of the wonderful work everyone’s doing – but let me know if you think we should change course at any time. This is only a proposal. 😊

With this idea in mind, here’s an agenda that outlines three examples that show how a protocol could *in theory* be used. We can discuss each of these and maybe propose different or additional examples and add to the list. The idea will be to select one example and write a specific protocol that could be used for that example as an exercise.

*** Agenda for Friday’s meeting ***

agenda+ Protocol for a standard where test results vary so you can't create a test case. We'll be discussing clear language as an example.
agenda+ Protocol for how to perform testing against a user process. We'll discuss screen reader testing as an example. **(Definition: User Process - Series of user actions, and the distinct interactive views that support the actions, where each action is required in order to complete an activity).**
agenda+ Protocol for how an organization can do user testing

Meeting info:

The Zoom teleconference data is provided at this link: https://www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000


We will be on IRC using the W3C server at https://irc.w3.org<https://secure-web.cisco.com/1bbHg1R0NpE7wu_f231-AnlS80Y-y5vJTvdOEWJENhb8A72iXd8LCnf3ggBw4-smyidfgNxC8x8umv7E05ehkUF2bZBz2YRztLLv4RKaBjPX52PKUDxmvfMGEBjoozskkcieYlkp03z0RNZpT4OYcOd4hVzq8R7ZxdOFKZWkBKST8tH_692bct2eWCZGqxEH2CqP3AYWJvaQCFfDc0IutM6Rj-U09KIPjPRnc7FV26Wdj10BaRhnKyirh5JjBQBiUijZJ9qa0yX2qAvF17eId_iMnLYZJUiOJFMccNsS6Lx1I4XZzBbvOxVip14Ng8wGLJfXvx15u76cOB7dAeH7XvgpoWs6R_tsGa3dqAcv4R_28w7gc_bd-NL4b_nyhs9ocwFRphYixX3lTUlkEH0Nokh_o6hHppEenQieUgyV9aZx_s1E0SmDtT5e5jzWNn35PIXxCXehg937DHDABmN_O-g/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Firc.w3.org%2F__%3B%21%21ACWV5N9M2RV99hQ%21ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ybOl3ZsYw%24>, in channel #silver-protocols

Where to find more information:

These and additional details of our work, including minutes, current, and archived draft documents are available on our subgroup wiki page here: https://github.com/w3c/silver/wiki/Protocols<https://secure-web.cisco.com/1TNW89z_MY_0wX1ZYm6hv5HS9mcihP85_ndBNUs4jpmXJbINRoeiL2PAOJpa48gu7ggqcaxvHcZwyDmk_ECpjpiMUgaWKtTr-m-I74FImn-3ikVIwa86LJPcN8hwZc-zIxld8vOpkMKuj8DOIBzEUkQSxpVAvG6QM5k6rIUvn5UWwVtD3jGjg4p4eOcnHvm-2wNCUEoe48xjGufzTTwNa312ZZktk4lQkG_1ur6TxT6m-t7nG-EjZ1_aT5QnxP1W05e9-VshxCDA7_jFtbuOuymikRPeuyrQEfPtulxR0eFvM9cHs42Bph8gxKy_7XqZnk9hsFYgkllVLW9UDnpau9WUQxWq-mTlt7DDfafP__A9wQbrcnkexMJ4u0s13-sS8AB8_tjJLZnbK3RhPQRGxlyArwz-iCfIthrluv7hfCYMML5BTIFe9fyZlBm0AUKKItBgs41V67OKE5DRc9pGtGw/https%3A%2F%2Fgithub.com%2Fw3c%2Fsilver%2Fwiki%2FProtocols>


Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://secure-web.cisco.com/1KQT2HpwaDwcn28aItsAgaue8wdYCC4sZY0KSP7v8e0y6O80eA0OX9NHGS9tNCifShUdIgOZS2de9CbRQK5e5KzDipD8TtlpcbFV326enom54lQyEviimbvQmq9a0OOD-_SimOVL13U3QfRnxQtSNzinjnSyt5ZuLuvIz3s9ZXskuix69i0yU9_9ZLFdtrdOHL11citwZyYdUEeQvlTR6JwA7D0dmdUmpVBrrIUG3vEm82BPVuSjFwBHdEKtA_wzmHQEhOuxwkXnFGf7v3ZdZadgXH-rRChC0Arj7G7TUbi3k6xwP8SEkcWAgCCI9ZrzPKAEXIPACNovIUh1SUYMSSHgu1nEYnsQoeA3DyX3nmYrtTIeF6ozoE5jc-TzrSp5vtRJwyvuFD5l085Adrb33wQDhB4abYvPTOHoKcMEo5zMMHvuSu0WVePXVpN5lJ5WbOSXJ0EO8tLcjK9JtRIq-dA/https%3A%2F%2Fwww.accessibilityassociation.org%2Fs%2Fwascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.] [Digital A11ies -- Working Together for All]



--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"