Re: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Take a look at BBC Gel Accessibility:
https://www.bbc.co.uk/gel/guidelines/how-to-design-for-accessibility

The contents listing on the left provide the “steps” for the protocol.
I’ll be honest, I’m not thinking at all about how an org’s use of protocols will factor into their “score” at all.
I’m focused on their using the protocol to evaluate the accessibility of their outputs, doing the work to ensure they are delivering the most accessible outcome — for the subjective areas that aren’t easy to determine in the WCAG SC or automation. Using the BBC Gel Accessibility as a protocol, we would document how we used familiarity / give control / offer choice / add value, considered content structure, annotations were documented, and capture what was done for user testing. I imagine a report / essay / page that conveys what was done.
It feels obvious to me, and I hope that it will be clear if we go through the activity together.

From: John Foliot <john@foliot.ca>
Date: Thursday, April 7, 2022 at 10:54 AM
To: Jennifer Strickland <jstrickland@mitre.org>
Cc: John Foliot <john@foliot.ca>, Jaunita George <jaunita_george@navyfederal.org>, "public-silver@w3.org" <public-silver@w3.org>, "w3c-wai-gl@w3.org" <w3c-wai-gl@w3.org>
Subject: Re: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan
Resent-From: <public-silver@w3.org>
Resent-Date: Thursday, April 7, 2022 at 10:54 AM

Hi Jenn,

Yes, you are correct.

I have previously stated that it is my opinion that the lack of progress on how scoring and conformance will work in WCAG 3 is the primary reason why, by appearances, most of the sub-teams are today stalled and do not seem to be meeting or making any significant forward movement. (I've had to step away from participating on calls for the time being, but I do read meeting minutes and continue to follow multiple mailing lists at the W3C. However, my impression today may not be completely accurate due to my no longer joining every call, but I do not believe anything has substantially changed since prior to my stepping back a bit.)

We're stuck on how all of this will piece together, because it seems to me the goal of how to reach Bronze, Silver, or Gold has not yet been specifically accounted for or even defined. It's a great goal in theory, but we've got zero evidence of how that will work in practice (because, I assert, the only way we can differentiate between 3 levels of conformance - or perhaps 'maturity' is a better way of expressing it - is to have a common unit of measure - the 'point'.)

Our draft spec, unchanged now for more than a year, has nothing but hand-wavey goals<https://www.w3.org/TR/wcag-3.0/#conformance-levels> when it speaks to conformance, and while it does speak to "scoring" in passing, after that remains incomplete ("This priority is reflected in the scoring system, which does not allow for errors along the paths needed to complete processes but allow for some accessibility errors outside process completion."). What scoring system? And, at this point is "path-based evaluation" still a thing? If yes, what progress has been made on that in the past 15 months? (It's also worth noting that there is a backlog of comments<https://github.com/w3c/silver/issues?q=is%3Aopen+is%3Aissue+label%3A%22section%3A+conformance%22> in our Github Issues related to conformance that we've seemingly avoided responding to for more than a year now.)

I have raised this concern previously (and more than once) and I am loath to continue to harp on about this further, as in every instance when I've raised this concern I've been told my questioning about points, scoring, and conformance is 'too early' - that somehow once we've got everything else figured out, the 'points piece' will all just magically come together. I have personally disagreed with this response from the onset, and I leave the evidence in front of us to judge whether I am seeing this correctly today or not.

JF

On Thu, Apr 7, 2022 at 10:12 AM Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>> wrote:
Also, aren’t we supposed to set aside the idea of “points” because WCAG 3 still needs to define scoring & conformance?

From: John Foliot <john@foliot.ca<mailto:john@foliot.ca>>
Date: Thursday, April 7, 2022 at 9:59 AM
To: Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>>
Cc: John Foliot <john@foliot.ca<mailto:john@foliot.ca>>, Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>>, "public-silver@w3.org<mailto:public-silver@w3.org>" <public-silver@w3.org<mailto:public-silver@w3.org>>, "w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>" <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Subject: Re: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Juanita writes:

> Would this replace any testing to validate that a company has met 1.2.2? Or, if they say they’re adopting this protocol, does it just add a bit of points to their total score, but they still have to meet the “testable” aspects of the guideline or SC?

Putting aside the fact that SC 1.2.2 is a WCAG 2.x requirement, I believe I still understand the question.

Looking at SC 1.2.2, it normatively states, "Captions<https://www.w3.org/TR/WCAG21/#dfn-captions> are provided for all prerecorded<https://www.w3.org/TR/WCAG21/#dfn-prerecorded> audio<https://www.w3.org/TR/WCAG21/#dfn-audio> content in synchronized media<https://www.w3.org/TR/WCAG21/#dfn-synchronized-media>, except when the media is a media alternative for text<https://www.w3.org/TR/WCAG21/#dfn-media-alternative-for-text> and is clearly labeled as such." This is, by my reading, mandating the provision of captions, but does not speak to the *quality* of captions. Today, this means (or at least could be argued) that, for example, YouTubes auto-generated captions are meeting the minimum requirement of SC 1.2.2:  captions ARE being provided, but as we all know in practice, sometimes those "craptions" are far from useful. But, from a narrow and strict reading of the normative text, SC 1.2.2 does not make any demands on quality.

Adopting something like Caption Key as a protocol starts to introduce the idea that *quality* captions are what is really needed, but recognizes that measuring quality is subjective. However, if you apply the guidance outlined by Caption Key you stand a far greater chance of creating quality captions, and so when it comes to scoring, I will reward you 'some' points for providing captions (based on ACT-style rules), but 'more points' if you also apply the principles of Caption Key to your output. (Requirement + Protocol = Final score: Protocols augment Requirements)

(In this regard, EO's alt text decision tree is potentially a basic and rudimentary candidate protocol, as it seeks to 'educate' content authors *how* to arrive at a useful and quality text alternative).

So it's not either/or, but rather Good, Better, Best...

JF

On Thu, Apr 7, 2022 at 9:21 AM Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>> wrote:
Would this replace any testing to validate that a company has met 1.2.2? Or, if they say they’re adopting this protocol, does it just add a bit of points to their total score, but they still have to meet the “testable” aspects of the guideline or SC?

And if saying that they followed the protocol gains them points, is there any limit to how many points they can gain? Or can an organization say they’re following all sorts of protocols and bump their score up to any level?

Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://www.accessibilityassociation.org/s/wascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.] [Digital A11ies -- Working Together for All]

From: John Foliot <john@foliot.ca<mailto:john@foliot.ca>>
Sent: Thursday, April 7, 2022 8:58 AM
To: John Foliot <john@foliot.ca<mailto:john@foliot.ca>>
Cc: Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>>; Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>>; public-silver@w3.org<mailto:public-silver@w3.org>; w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>
Subject: Re: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Might I also offer as another potential Protocol something that Wilco asked about a week or so ago... and that is Caption Key (https://dcmp.org/learn/captioningkey<https://secure-web.cisco.com/1SolsNGoqdTXSlh7NxIbGgIAUfoY2TE6u9k_xmMrDem6XdvWesSz8_MdIKjGKveFP_C0E1rqaid_ZWvIHsS0hntubQgzUfOPLik2pMzhvqlcKfuX31A1ykp3CPaMNfMhv75RzEo8xyp7KyyZ0f40hiJUYuBSRBcSIlHG8A3odEjWFirMupF2i9JjhDpcnfdvASQuBp2W42Zu4vvm6n1owH2jH7m5Cx4r_FNKDhlZ6zoWCeS48XUxoCJZntr_5nKUj22aOrKBtEU-dA2_gVvePAinc-CbsRt2bxdaIB8O09ch-0zpx46SkOmb9yztJnjtL22A3BFX87SgI_8KvlyVOA2u0FYhVZUT-5kkKNVdfLGapWf7mvy0wnFgcslwXLiklYr628mwo7UCyG8moCO13O-GcB3hPi7wnOmR93bZGini29R23fpnXtodXQlfTI01m7Ml4vvdbkZJtmzxyFfPCgg/https%3A%2F%2Fdcmp.org%2Flearn%2Fcaptioningkey>)

As part of that larger site, it outlines "Elements of Quality Captioning<https://secure-web.cisco.com/184ycatIGNo0NAq8sjTpwk1uUdC40OW5vT-QiFBNdqDWg2zRW2VeT--TIDnurzKwP6gVr3Zy-AYMT__jW_5xWBeAIgoorOGNRQdB8VVUZYJlAmx_BnmA3tqtRgDDSseBOIg4BdPuaU-pz5t64a431IWoSwvq9uhBx6bcUvDDQfbDI2WwLwjCITiCGGvGXDsdKxVqge09P5_wXCF5XdnfzDSAiWw0hKCf3LZiHojKcHwM-6ChR_0G-yQeMv5TsniHqabULuSlD2Hfl5laIFQxDTvakckDlw6hO2czCBoIU6c3a9vEj72imgFTDWhj8Dupp6gCMhqVgUbZM4kjx4IhjLwbKHcnBjk4wI58dJnmj_o-60DLOP9Av5QW8AUoBy389znhRu7EUV6uFY0nGqOr2pWGe9cw5THxQagcgOPiz_9oa2RzQrRsYUCI6FNI6VvvKk0tsaxX1cGem01KgoLHoTQ/https%3A%2F%2Fdcmp.org%2Flearn%2Fcaptioningkey%2F599%233>", which includes the following expected outcomes:
•       Accurate: Errorless captions are the goal for each production.
•       Consistent: Uniformity in style and presentation of all captioning features is crucial for viewer understanding.
•       Clear: A complete textual representation of the audio, including speaker identification and non-speech information, provides clarity.
•       Readable: Captions are displayed with enough time to be read completely, are in synchronization with the audio, and are not obscured by (nor do they obscure) the visual content.
•       Equal: Equal access requires that the meaning and intention of the material is completely preserved.
(I find it interesting that the first bullet point sets out a "goal" but not a mandate: they recognize that 'errorless' simply cannot happen 'always', but that it should not put off entities from striving for errorless captions - only that the odd error is not "the end of the world". I wonder aloud if this is or would be part of a protocol's expectation - strive for perfect but accept less-than-perfect? Additionally, bullet 2 seeks "uniformity" but comes up short in defining what uniformity must look like.)

However, on the plus side, the 'authority' of who authored these goals is beyond question (well, at least to me), and so as another element of defining a protocol, we could consider who authored the protocol in the first place - and I believe we could certainly be the gate-keeper on which protocols would be in scope by our spec (this avoids any entity making up their own stuff in an attempt to 'gain points').

JF

On Thu, Apr 7, 2022 at 8:44 AM John Foliot <john@foliot.ca<mailto:john@foliot.ca>> wrote:
Hi all,

> At the request of the COGA team, let’s avoid using “Making Content Usable” as an example of a protocol.

Really? Where and when did that request come to this group? I have to admit, I am somewhat astounded that as the larger group is seemingly moving closer towards ACT-type tests (agreed-to and in our charter) that COGA now think that the document that emerged because they could not author their needs to fit the WCAG 2.x structure will be malleable enough to fit into Methods in WCAG 3.
Oh well...

> Plain language is a good example, especially for clear language when applied to content in English

Hmmm... this statement seems to imply that plain language and clear language are not the same. Can you elaborate more on this please? What are the differences, and what (if any) impact does that have on content created?

> I would imagine that companies of different sizes would implement protocols that would apply to companies of their size

Protocols? (as in plural?) In the context of user testing, I am unfamiliar with even one specific document that could serve as a scalable protocol today, and yet this seems to suggest we could emerge with multiple protocols (based on company size) for any given topic/need. I fear this introduces yet another barrier: defining 'size' and the breakpoints required for that. Is size based on head-count? Revenues? Impact on society? Something else? It's relatively easy to contrast a giant (IBM) against a small company (https://www.fullcycle.ca<https://secure-web.cisco.com/1h4yFLHRg-y1ttKNNIgA498RB7XzGlDDXB47KZqYeCYOVyUtcgZTvDvoxeTzX9BwM2ooFILWJkVF9Gx7jHVmYYjLqeA6jIuDv7lw_PTNdi2xmQNY-_n_6DZkiMHJ7c62xbYx1ugAhFrLRjKn2r9EZ9S3l3LqFcBjM7S8-P7BVpv8bXXqqzBjPs1L_SNkMxxgoA8XT0Pph6s7vXYJTsMJKRt8wJH8pEl4qoi0dkOBkQ9FjKh21WwGNTe_NkQt6kxulK5H78F5ceCdgO9MWGcf5hhf2XSlp2USH7c-18jQaloLSlVHClGgf0FdZQC3TqkxjYIcGWRy99civ4iMIRRt9FLnMTxpLfL2pmX-2WBavSev4621_wSWLDTCvopOxnQ2KLCCUwxDpbeK7Uw4VUS9GV0jAqfQD1us5YymBbTL1ZxaIYFkLSeHz967upGIdHg67lyGyBsTzoEaQIKHJiYte1A/https%3A%2F%2Fwww.fullcycle.ca>), but what about companies "in the middle"?

But, this also seems to contradict the idea that as a first pass, we will look at plain language (as defined at plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov>). That is a single document (protocol) that is applicable to all sizes and shapes of web sites. So as an open question, do others believe that we might have protocols based on the size of an entity?

> I am a little concerned that it’ll be difficult to make progress if we focus on a definition without giving concrete examples about how protocols will be used.

Indeed. I personally have always felt that specific "outcomes" is counter to the idea of a protocol - I suppose I am focused more on the concept of an "Operational Protocol". (I found this definition<https://secure-web.cisco.com/1Sn1j7ZWFJk0X8C1pExa8nXPSDo6s3-FBuNqgsjTgiul_JR_1CjaPX2aW0em7SuEdbElERMdg_V4BSH2suWbXQp9wakZHHT4_g4prljVb8bDSp31UTVzR4L0peXH6McUIhERPE9WOQFNPLIFl9VaqsUDGJz-hhWpvD3sDC4QbgkR82yGzCZsPff0-WIGkB58jYi9TIVJUeaaLrleO4g2borheYF-FhCfxEKL7z9noA5VKKk-QFDWFiyeGYwUagaDOzOMyWi8vHcJ136HH7uZBFKQdfRrLpSponsaFYZPH5Qe81ZCkmuOhtOdQU7osGDoWuUn3JuKJgsggOLVwZtkqwHxxzK4h8C3ru-B-7QYohMTSArf5g-WxrstBO56nuQqzAhlIzRsqZV34esvqlWq4x59kWmO_4oyc2cQ7DjtX_YNTpViKtDxwOGCehT3GZIAR5yCruDKjGsp1mYwr8uHFig/https%3A%2F%2Fwww.lawinsider.com%2Fdictionary%2Foperational-protocols>, used in the context of municipal governance: "Operational Protocols<https://secure-web.cisco.com/1Sn1j7ZWFJk0X8C1pExa8nXPSDo6s3-FBuNqgsjTgiul_JR_1CjaPX2aW0em7SuEdbElERMdg_V4BSH2suWbXQp9wakZHHT4_g4prljVb8bDSp31UTVzR4L0peXH6McUIhERPE9WOQFNPLIFl9VaqsUDGJz-hhWpvD3sDC4QbgkR82yGzCZsPff0-WIGkB58jYi9TIVJUeaaLrleO4g2borheYF-FhCfxEKL7z9noA5VKKk-QFDWFiyeGYwUagaDOzOMyWi8vHcJ136HH7uZBFKQdfRrLpSponsaFYZPH5Qe81ZCkmuOhtOdQU7osGDoWuUn3JuKJgsggOLVwZtkqwHxxzK4h8C3ru-B-7QYohMTSArf5g-WxrstBO56nuQqzAhlIzRsqZV34esvqlWq4x59kWmO_4oyc2cQ7DjtX_YNTpViKtDxwOGCehT3GZIAR5yCruDKjGsp1mYwr8uHFig/https%3A%2F%2Fwww.lawinsider.com%2Fdictionary%2Foperational-protocols> means the administrative policies and procedures of an EMS System or that provide guidance for the day-to-day operation of the system."  For me, the key is the final phrase, "...provide guidance for the day-to-day operation of the system." )

Success, in that context, may ebb and flow when applied to specific output (sometimes results will be better than other times), but the operational protocol keeps the 'team' pointed in the right direction, and provides guidance and education when faced with broad (or even specific) decision making.

A Concrete Example:
(Remembering that my proposal awards entities for adopting protocols, and not the output based on that adoption... And that a large part of the key is the public assertion, using a specific reporting format.)

Large Company: With offices in 3 continents and an employee base of 18,000+ employees, the XYZ Widget company adopts the guidance found at plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov>. That guidance sets forth 8 specific outcomes:

  *   Write for your audience
  *   Organize the information
  *   Choose your words carefully
  *   Be concise
  *   Keep it conversational
  *   Design for reading
  *   Follow web standards
  *   Test your assumptions
As to 'how' the XYZ Widget company applies and 'meets' this protocol, I will again offer a strawman example (based on the public attestation piece):
"The XYZ Widget company's editorial team have adopted the plain language requirements found at plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov> into their internal authoring guide. All editorial personnel at the XYZ Widget company are aware-of and use this authoring document when writing materials related to the XYZ Widget company's products, offerings, and related corporate information."

This is written as a factual statement, without actually offering any examples or 'proof' that (for example) every member of the editorial team has a printout of the plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov> site content on their desk, which they refer to daily. Yet, with little effort, 3rd parties *should* be able to find evidence that the principles of plain language are being used by the editorial team (sometimes results will be better than other times). In practice, here are two examples:

  *   Write for your audience
Strawman: Because of the size of the company, aspects of internationalization will be applicable here. So, for example, when writing for an international audience, awareness of cultural norms and taboos will impact editorial decisions. Additionally, given the size of the organization, there are editorial teams located around the world, and the company has already established an internal authoring guide, (similar to the design guide they give their clients when their clients wish to use the company's logo in localized advertising efforts).
  *   Organize the information
Strawman: When looking more closely at this goal, it states the following: Make it easy to follow, Add useful headings, Have a topic sentence, Place the main idea before exceptions and conditions, Use transition words, Use lists.

This is all broad guidance, but not easy to 'measure': for example this document states "use lists", but then stops short of when, where, how many, etc. However I think we all know it does not mean

     *   make
     *   everything
     *   a
     *   list
(...and the requirement falls short of defining what type of list: bullet, ordered, or definition. I could choose one list type, and Jennifer could choose another list type, and who is to say which choice is better than the other?)

So 'subjectively', before an entity can 'meet' this bullet point, education and understanding of when, where, why and which types of lists are preferable needs to happen (and again, for the XYZ Widget company, the i18n context is applicable)

Small Company: The Smith Family Yard Services is a family-run business that has been in operation for 23 years, providing yard maintenance and landscape design services. Founded by Fred Smith in 1999, the company includes Fred, his two sons Bob and Joe, and Fred's wife Jane, who manages the office and is responsible for all of the book-keeping, customer service (answers the phone), and has been tasked with keeping their online presence up to date. Jane uses GoDaddy to create and maintain the family business website.

Attestation:
"The Smith Family Yard Services strives to meet the requirements laid out at plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov>. As part of that commitment, the maintainer of The Smith Family Yard Services web presence has taken the online training provided at "The Essentials of Plain Language" - a nine part training<https://secure-web.cisco.com/1et8JlmTaMTWIDBW3PLZOiG7QS5C8XgAPCbB1V7jLgrJlOxkbVFazwno9FEcWxzY1DU7QGz5sgm9y_JcEZvoR-nWRyk72axHduUMzKx3RpTzmk5oD2Fcx8HrayPLy9FgJm4fkGnu5e2u-ak7tiycsdU-Qx_pwEW3aIFGee31IeOQh1YGqgzDm55BFwIzGjeDYCCFjKPnReWQo6hnFRuD0kF7u1T2IyHdXcdPF1REFjmLAXAkgKN4WCZoB5KTAo8KvMjQqhmGbAJMJpO7p0Acs2KjpEIxrt-dkhdqTjmgUKuNQlcrq3WBqP5o2GmwSg3PAgx3aRr45ltQb4CGXvKB4ehtaEpJrTlmDdg-0hijVT6kE0GnTdC95hEgVNsQD7wS4PElaHW09ourI_Y9bwiCwCXfX-VGHEf5xDoriUuXGr9dQzg0qFxz9P89CmlI61htbYO4_X0MNZUM9PV45DtD-FA/https%3A%2F%2Facademy.govloop.com%2Fwatch%2FhDzHyqdB4T7K3fjbvuGk8B> that covers plain language principles."

Now, yes, I have concocted these strawman examples to serve to illustrate my perspective, but clearly the two example businesses are light-years apart. Yet, when you step back, both of them can, in a size and scale-appropriate way, adopt and attest to the adoption of plain language as outlined at plainlanguage.gov<http://secure-web.cisco.com/19tG_vx_KpKkWWkzGEOzlmlMqucQwi3wotDfUDEh4p-5zjyyPfKaGIqJZXuy0_0S9_L1uii7K7XkLxEYK9oQwG93FAp5xtxlU12hEkkOvOXDAsKH7HVafYYQDce6RlqR2L886P6PhPOFs6JIomDc80wYJe4GeqMECpxxpyklXtMI9WKKF2C2pdBimpdINIB4nAKwMmFtyEKCbTSAyjmm4xO4vUPl8bJEl2VgiiVJvhdfr2Bx98pGXBW_e4BDMq0eUSIVStwGEHNVisBuW-32zDnQrU7uZgmavMUdS9WmN3ozVqiyJ-wi06iS_0MlN4d8LA-sirWNkEshr3zOR8O-ad7V-pEDOzaLNc0DSvG4O97n9FV8Wp_wycYj53PgsoPIzJsnkdpeAml5PZovq2c_lqcjACKrmu6RVQsixWaaJTdSPVU7i-QylkyVQPIcMMjDcsUkJj8kUvTjMG37vQLZkjw/http%3A%2F%2Fplainlanguage.gov>. And, as part of the attestation, there is a declarative statement that references education in both examples. The "How" of that education is completely different, but the net result remains that both companies are attesting to the fact that both are aware of the need/requirement for plain language, and have taken proactive steps to address that need - again in a size and scale-appropriate way.

I guess that this is the long way of saying that we cannot "measure" the output of either company against each other, but in both of my sample attestations, the idea of providing evidence of 'education' is an important 'proof point' (which the Maturity Model folks are using in their effort). And so, when applying a protocol against these two concrete examples, the commonality is in the approach to applying the protocol, and not the output that results from that protocol.

Respectfully,

JF


On Wed, Apr 6, 2022 at 9:27 PM Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>> wrote:
Hi all,

At the request of the COGA team, let’s avoid using “Making Content Usable” as an example of a protocol. It could imply that the information contained in there won’t be used in methods and other parts of WCAG 3.0, which isn’t necessarily the case. Plain language is a good example, especially for clear language when applied to content in English (maybe only in North America, not sure if it applies globally to all English speaking regions of the world).

I am a little concerned that it’ll be difficult to make progress if we focus on a definition without giving concrete examples about how protocols will be used. It can be used to measure inputs and not outcomes in any of those scenarios identified in the agenda and it’s hard to picture what would make a good protocol without understanding how it’ll be used exactly.

For user testing, I would imagine that companies of different sizes would implement protocols that would apply to companies of their size and would make sense for their business. User testing can be approached in theory in similar ways, however, no matter an organization’s size – you’ll still need to recruit a panel, define scope, etc. An organization, could then, in theory, use a protocol to help them create that process. It could also define a process for adding insights gained from user testing into an organization’s backlog or define ways an organization can implement/categorize feedback.

With the user process example (like with screen reader testing), you would likely have an organization adopt guidance about how to perform screen reader testing generally that would include how screen readers work, how to test different kinds of functionality and such and then some expected behavior. This would be general guidance that can be applied to multiple guidelines and methods and would represent some general best practices that can be applied at scale. It’s likely that different kinds of folks using screen reader software for testing will get wildly different results (as a person who uses screen readers every day would have a different perspective than a QA tester), but the protocol can put some structure around that to help organizations achieve something more consistent. 😊

Is there a single particular use case that everyone is moving towards? If so, we might want to define what that is and that might help us move forward. If it’s only for agenda item 1, for example and that’s the consensus, then we might want to define which (exactly) standards are so subjective that a protocol would help an organization achieve an outcome:


  1.  Would it only be clear language?
  2.  If there are other standards that would apply, would any of them come from WCAG 2.x or only in some of the newer outcomes being defined in WCAG 3.0?
  3.  In either case listed in two, would it be helpful to create a list?

Seeing the universe of cases where protocols may apply could be helpful for defining the requirements for an acceptable protocol, but that definitely could just be my own opinion.

What do folks think?

Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://secure-web.cisco.com/19XjS-3jc60o9w2HMPMzCN__NCYkXtVJH6ie09GLZzURiHwp_JwhCppF1JlsZws5RiA6Z1S22Dy1jtnnwGYwiNNmauRqceo82Aq0NxK7_EV7qyU0_lW63SUh1DXcg4ZE9h-mWq--6ZnPJYP_a5wqCvqXgLiNuyZ-i4YT03CVCWD31nsjXrQ20EmPKM4nLIQI_RBt6RpkqZ41BZUwO1JFoXjo9Wb_GN4R-ju37PMYiMX7ygaEcwhCUaWBlNMpMi8TM4QibTNtRYHSSJt86DRXygB4_agUG4nSr-AaSIdbJ1AeDzM0DZDfSVkNxK3AwKmyUosK9ftoB0PUfsOcaj6s3vvouKdN8OR9jDfqJzmIivE5VfMxLgRqt1rnKX-mpd_7WqGq2y2fXPGbzd7V8rfeMzkLS5Yk2Yw84s861XdMcKsglPBOYtyYMyZq6QMJFKkDvT__axwsbSUC08WYJQdAUvQ/https%3A%2F%2Fwww.accessibilityassociation.org%2Fs%2Fwascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.] [Digital A11ies -- Working Together for All]

From: John Foliot <john@foliot.ca<mailto:john@foliot.ca>>
Sent: Wednesday, April 6, 2022 7:40 PM
To: Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>>
Cc: Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>>; public-silver@w3.org<mailto:public-silver@w3.org>; w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>; Benjamin Feigel <Benjamin_Feigel@navyfederal.org<mailto:Benjamin_Feigel@navyfederal.org>>
Subject: [External] Re: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan

Hi All

A strong +1 to Jenn's comment, and a reminder that we had originally agreed that "Protocols measure inputs and not outcomes." (7 January 2022<https://secure-web.cisco.com/1DnoYE9Nk9im8xHHdlEwNUiP3J3o9S1NJgh3iKPH60aqH7Vjbpo5op7zW0HCvTTxGcJj0LFeqnZcHmTu3YQrIOxsRl7XIGISptTBogKMGL6AyIiamnm0rgm6IwTI2zwecIi7mYNxVFInR2lcuj9jY1lVhdVnaZn3IiRzaDPoMLhLkW1_F1K9PoWXm5PtCk_N8HWwVwPHtNOqkBXNsz88651maxViYSm0z7eRmsy2U963vxFGU2nSDJGFDrdSNUZv2cRhCMc2VWC-zjb7qwzJY6SlpuVLuy4olwD0TRVOC4L7qkzX4CJmXRGP9H1l1Aatuc3XHajIjYOGANt1dfd2pa4fmYBtQzo9SEWMXj9WDtnCdkOkXpSkZecsY7FU4W8Icc6h1hNCH9unJp1ayFtJ1RKS7zg87-SQzeDa0XEtAHuk6YhzCDUoPKeLRdQIBIST1ZW8EMPMfo1tmqfQvxKJMfA/https%3A%2F%2Fgithub.com%2Fw3c%2Fsilver%2Fwiki%2FProtocols%23key-decisions-agreed-on-with-date>).

Yet, as I read the agenda, it still feels like you want to look at outcomes. But this is where I think we get stuck - because determining outcomes for things that cannot be measured using ACT-Rules-like requirements will remain subjective and is the real problem.

Why a problem?
Because if/when you ask an entity whether they succeeded or not, *of course* they will claim that *in their opinion* they have. Yet, in the context of legislation, *of course* the litigant will say the contrary, that they haven't. You're still trying to measure outcomes!

Attempting to measure these points of view cannot (I assert) be measured, for the basic reason that they are opinions. (...and as I used to tease my daughter, "everyone is entitled to a wrong opinion" ;-) )

Continuing:

  *   Protocol for how to perform testing against a user process. (JF: useful, but not in the context of actually making the content more accessible, a 'protocol' like this would outline the steps you need to take when testing, but does nothing to guide or inform content creators. And how would a user-process be scoped, by whom, and how/why? Every time you introduce a potential fork in the user-path [clicking on a help icon for example] you have to build out your 'flow' to account for that... it doesn't scale! And "happy path" testing will usually 'pass', by my experience it's when the user has to deviate from the happy path that things start to go sideways...)
  *   Protocol for how an organization can do user testing (JF: Again, a useful set of guidance, but it may not scale either: Susan's Flower Shoppe (with 3 stores in the tri-state area) will simply not be in a position to do the same type of user-testing as Amazon or Facebook, and attempting to determine any kind of stratification (different processes for different sized orgs) will also introduce a real quagmire (where do you draw the lines, and why?). I think the best you could ever get there would be an assertion that user-testing was performed on [date] for the following flows [list flows] - but... does that testing then absolutely ensure more accessibility? (NOPE) The real win is taking results from user-testing and applying that knowledge in the next round of development. The real value of user testing is what you learn from the testing, and not the actual testing itself.)
Might I then respectfully suggest that rather than kicking off asking "how we can use a protocol" that instead we seek consensus on "what makes a candidate protocol acceptable for use in WCAG 3".

If we remain true to earlier agreements (Protocols measure inputs and not outcomes) then I will suggest that a key commonality would be that it represents outcomes and guidance geared towards the creation process, and NOT the testing/evaluation/measuring process. Shift Left in practice!

I personally envision adopting protocols as essentially promising (publicly - for the accountability piece) to do the requisite research to achieve the outcomes as described, and I argue that winning that education battle is worthwhile in-and-of-itself.

So when Making Content...COGA or PlainLanguage.gov outline Outcomes and then explain the issue and strategies that individual entities could apply *in context* to their content, they are in fact 'teaching' - and I assert THAT is the real value of Protocols (as I envision the definition of the term related to WCAG 3).

So with that definition, now Susan's Flower Shoppe and Amazon could both "learn what makes Plain Language" and then apply that learning to their content IN CONTEXT - scale is no longer a problem in the traditional sense (although it will be harder for larger orgs to remain consistent - but they will also be in a better position to have policies and processes in place due to the size of their org)

JF
(who hopes he can join the call Friday morning... stand by)





On Wed, Apr 6, 2022 at 4:05 PM Jennifer Strickland <jstrickland@mitre.org<mailto:jstrickland@mitre.org>> wrote:
Hi Jaunita and all,
Thanks for communicating an agenda ahead of time. I think this can be very helpful and avoid using the meeting time to agree.
Regarding #2, “Selecting and writing one protocol from scratch as an exercise,” I don’t think we would write a protocol. We could document how one might document using a protocol.
Previously we said we agreed to use PlainLanguage.gov as the protocol and then ended up evaluating what the US Department of Labor documented for their efforts to meet, as I read it, the Plain Writing Act, which is related but a law rather than a protocol. Now there’s a proposal to test “Protocol for how to perform testing against a user process,” using screen reader testing as a user process — but do we have a protocol to use?
Can we agree upon a protocol and site to test, go through the process of how a person might do that, as we previously agreed to do?
Thanks,
Jennifer

From: Jaunita George <jaunita_george@navyfederal.org<mailto:jaunita_george@navyfederal.org>>
Date: Wednesday, April 6, 2022 at 3:38 PM
To: "public-silver@w3.org<mailto:public-silver@w3.org>" <public-silver@w3.org<mailto:public-silver@w3.org>>, "w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>" <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Cc: Benjamin Feigel <Benjamin_Feigel@navyfederal.org<mailto:Benjamin_Feigel@navyfederal.org>>
Subject: [EXT] [Protocols] Agenda for April 8th, 2022 and Proposed Plan
Resent-From: <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Resent-Date: Wednesday, April 6, 2022 at 3:37 PM

Hi All,

At the last meeting the team mentioned that I’ll be stepping in as a co-lead for the Protocols sub-group. I’m really excited to be working with you all in this capacity and to do what I can to further this discussion. I’m new to this, so please don’t hesitate to reach out if I make a mistake or forget something – I’m always available by email and am happy to also set aside time to meet and discuss any questions or comments you have. I really look forward to talking to you all on Friday. 😊

New meeting time:

Last meeting, we agreed to move our regular Friday meeting to 8:00am instead of 9:00am EST. This means that the Protocols Subgroup will be meet this Friday, April 8th at 8:00 AM Boston Time (1400 UTC). I sent out an invite, but please let me know if you didn’t receive it.

Plan for the next few meetings:

To help us answer some of these excellent questions we’ve tackling, I thought we could focus our efforts in the next few meetings to:


  1.  Achieving consensus on how we can use a protocol.



After we’ve achieved consensus on that question, we can move on to:

  1.  Selecting and writing one protocol from scratch as an exercise.

This may help us structure our discussions and help us continue moving forward on all of the wonderful work everyone’s doing – but let me know if you think we should change course at any time. This is only a proposal. 😊

With this idea in mind, here’s an agenda that outlines three examples that show how a protocol could *in theory* be used. We can discuss each of these and maybe propose different or additional examples and add to the list. The idea will be to select one example and write a specific protocol that could be used for that example as an exercise.

*** Agenda for Friday’s meeting ***

agenda+ Protocol for a standard where test results vary so you can't create a test case. We'll be discussing clear language as an example.
agenda+ Protocol for how to perform testing against a user process. We'll discuss screen reader testing as an example. **(Definition: User Process - Series of user actions, and the distinct interactive views that support the actions, where each action is required in order to complete an activity).**
agenda+ Protocol for how an organization can do user testing

Meeting info:

The Zoom teleconference data is provided at this link: https://www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000


We will be on IRC using the W3C server at https://irc.w3.org<https://secure-web.cisco.com/1bbHg1R0NpE7wu_f231-AnlS80Y-y5vJTvdOEWJENhb8A72iXd8LCnf3ggBw4-smyidfgNxC8x8umv7E05ehkUF2bZBz2YRztLLv4RKaBjPX52PKUDxmvfMGEBjoozskkcieYlkp03z0RNZpT4OYcOd4hVzq8R7ZxdOFKZWkBKST8tH_692bct2eWCZGqxEH2CqP3AYWJvaQCFfDc0IutM6Rj-U09KIPjPRnc7FV26Wdj10BaRhnKyirh5JjBQBiUijZJ9qa0yX2qAvF17eId_iMnLYZJUiOJFMccNsS6Lx1I4XZzBbvOxVip14Ng8wGLJfXvx15u76cOB7dAeH7XvgpoWs6R_tsGa3dqAcv4R_28w7gc_bd-NL4b_nyhs9ocwFRphYixX3lTUlkEH0Nokh_o6hHppEenQieUgyV9aZx_s1E0SmDtT5e5jzWNn35PIXxCXehg937DHDABmN_O-g/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Firc.w3.org%2F__%3B%21%21ACWV5N9M2RV99hQ%21ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ybOl3ZsYw%24>, in channel #silver-protocols

Where to find more information:

These and additional details of our work, including minutes, current, and archived draft documents are available on our subgroup wiki page here: https://github.com/w3c/silver/wiki/Protocols<https://secure-web.cisco.com/1TNW89z_MY_0wX1ZYm6hv5HS9mcihP85_ndBNUs4jpmXJbINRoeiL2PAOJpa48gu7ggqcaxvHcZwyDmk_ECpjpiMUgaWKtTr-m-I74FImn-3ikVIwa86LJPcN8hwZc-zIxld8vOpkMKuj8DOIBzEUkQSxpVAvG6QM5k6rIUvn5UWwVtD3jGjg4p4eOcnHvm-2wNCUEoe48xjGufzTTwNa312ZZktk4lQkG_1ur6TxT6m-t7nG-EjZ1_aT5QnxP1W05e9-VshxCDA7_jFtbuOuymikRPeuyrQEfPtulxR0eFvM9cHs42Bph8gxKy_7XqZnk9hsFYgkllVLW9UDnpau9WUQxWq-mTlt7DDfafP__A9wQbrcnkexMJ4u0s13-sS8AB8_tjJLZnbK3RhPQRGxlyArwz-iCfIthrluv7hfCYMML5BTIFe9fyZlBm0AUKKItBgs41V67OKE5DRc9pGtGw/https%3A%2F%2Fgithub.com%2Fw3c%2Fsilver%2Fwiki%2FProtocols>


Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://secure-web.cisco.com/1KQT2HpwaDwcn28aItsAgaue8wdYCC4sZY0KSP7v8e0y6O80eA0OX9NHGS9tNCifShUdIgOZS2de9CbRQK5e5KzDipD8TtlpcbFV326enom54lQyEviimbvQmq9a0OOD-_SimOVL13U3QfRnxQtSNzinjnSyt5ZuLuvIz3s9ZXskuix69i0yU9_9ZLFdtrdOHL11citwZyYdUEeQvlTR6JwA7D0dmdUmpVBrrIUG3vEm82BPVuSjFwBHdEKtA_wzmHQEhOuxwkXnFGf7v3ZdZadgXH-rRChC0Arj7G7TUbi3k6xwP8SEkcWAgCCI9ZrzPKAEXIPACNovIUh1SUYMSSHgu1nEYnsQoeA3DyX3nmYrtTIeF6ozoE5jc-TzrSp5vtRJwyvuFD5l085Adrb33wQDhB4abYvPTOHoKcMEo5zMMHvuSu0WVePXVpN5lJ5WbOSXJ0EO8tLcjK9JtRIq-dA/https%3A%2F%2Fwww.accessibilityassociation.org%2Fs%2Fwascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.] [Digital A11ies -- Working Together for All]



--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"


--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"


--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"


--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"


--
John Foliot |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |
"I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"