Re: [External] Re: [Protocols] Minutes for March 4th, 2022

Hello,

I concur with Jennifer that the protocols work is specifically for those items that can’t be assessed in another way. The purpose of the conceptual matrix<https://docs.google.com/presentation/d/1b5xHQWBzoYdKp7BfPgIUBCpz-yaDOx_kSq_HlQxcFh0/edit#slide=id.g115ec01aa81_0_57> we have been discussing is to help us talk about these types of differences and I am glad we are having a productive conversation. I’m pasting it below for reference.



Constant

Condition

Test Case

Protocol

Smallest Unit

Most concrete, objective, repeatable, prescriptive





Most abstract, subjective
Least prescriptive

View









User Process









Aggregate










I believe providing different types of tests based on the level of subjectivity with different evaluation criteria requirements allows us to expand coverage but still provide a way of evaluating consistently across organizations. If we want to stay with pass/fail evaluations, the criteria naturally become higher level as you move to the right. We move from “Did your result match our requirement?” to “Did your result match experts’ requirement?” to “Did your result match your requirement?” to “Did you complete a process to improve your results?”

I think we can add more subjective criteria without putting current progress accessibility at risk:

  *   if  we retain the more objective and repeatable criteria (constants and conditions in the matrix), and
  *   keep them distinct from the more subjective and less repeatable criteria (test cases and protocols in the matrix).

At least how I see it, the more subjective tests supplement the more objective tests – they don’t replace them.

Maybe the more subjective types of tests) lead to higher rating levels (test cases for silver and protocols for gold) or maybe we take a different approach.  Jake’s example provides a possible way to capture some information about protocols that can be compared, even though it is not at the same level of detail as the criteria.

Kind regards,

Rachael

From: Jennifer Strickland <jstrickland@mitre.org>
Date: Friday, March 4, 2022 at 2:02 PM
To: Jaunita George <jaunita_george@navyfederal.org>, jake abma <jake.abma@gmail.com>, "Bradley-Montgomery, Rachael" <rmontgomery@loc.gov>, John Foliot <john@foliot.ca>
Cc: "public-silver@w3.org" <public-silver@w3.org>, "w3c-wai-gl@w3.org" <w3c-wai-gl@w3.org>
Subject: Re: [External] Re: [Protocols] Minutes for March 4th, 2022

Hello,
As I understand it, the purpose of WCAG3 protocols is to define a method to show due diligence on guidelines that cannot be quantitatively measured. The reason an org might choose to use a protocol is to show they made an effort to evaluate the non-quantitative standards. If the guideline could be measured, it would be a WCAG method. Not everything can be assessed in Boolean pass/fall. However, there are questions people can seek to answer to evaluate the degree of accessibility of an implementation.
Protocols suggest that a set of questions and assessments to evaluate will reveal the degree of accessibility. By “showing their homework” it shows that they made the effort. There will always be a risk that it isn’t accessible to someone, but performing the activities to assess shows the effort was made to ensure the org delivered an outcome that is as accessible as possible, as is the case with WCAG standards anyway due to the evolution of tech.
Protocols are only for WCAG guidelines that cannot be quantitatively evaluated, that WCAG hasn’t provided an objective evaluation method. So, without protocols, there is no way to hold anyone accountable for the subjective evaluation criteria. By demonstrating the due diligence to ensure an experience is accessible, that would count. Once we evolve technology to a degree where everything can be objectively evaluated, protocols would not be necessary. The reality is that will likely never happen.
Automated accessibility tests can return a pass or fail, yet still be inaccessible, so how much do the objective measures really evaluate a screen? For example, we can say yes or no if alt text exists but we cannot determine the meaningfulness of the alt text. A protocol would provide a list of criteria to consider when deciding if the alt text is meaningful — while still remaining subjective. That’s where Plain Language (https://plainlanguage.gov) serves as an example. The protocol provides ways of evaluation and things to consider. If an organization identifies they used Plain Language as a protocol to go beyond the objective yes / no evaluation regarding alt text or clarity of content, that’s showing the good faith effort. I don’t think it is something that supersedes the objective measures of WCAG. It serves as an enhancement for where we are in the evolution of technology.
This is how I understand the protocols effort.
Respectfully,
Jennifer
Jennifer Strickland (they/them, she/her)
Senior Human Centered Accessibility Engineer
Collaboration and Social Computing (L178)
Email: jstrickland@mitre.org<mailto:jstrickland@mitre.org>
Cell: 571-319-2230
[Linkedin, Twitter, YouTube, Facebook and Instagram logos]


[signature_1635743481]


From: Jaunita George <jaunita_george@navyfederal.org>
Date: Friday, March 4, 2022 at 1:36 PM
To: jake abma <jake.abma@gmail.com>, Bradley-Montgomery, Rachael <rmontgomery@loc.gov>
Cc: public-silver@w3.org <public-silver@w3.org>, w3c-wai-gl@w3.org <w3c-wai-gl@w3.org>
Subject: RE: [External] Re: [Protocols] Minutes for March 4th, 2022
Hi all,

I’m a little concerned with having this be something that could be used to meet standards.

I feel like this would undo a lot of good work that’s taking place to hold organizations accountable if we try to make this a compliance standard or alternative to actually meeting the requirements. It also would make someone “compliant” if they show they’re trying to implement processes – which I feel is just going to result in lower accessibility across the board. Organizations everywhere have wonderful accessibility statements, but inaccessible products – and I think this could continue that worrying trend and lead to accessibility in name only.

If we go this route, then we should just make it part of the maturity model and not tie it to standards compliance. It’s important if organizations try, but far more important if they achieve results and results should always be easily measurable.

Jaunita George, JD, PMP, WAS (she/her)
QA-ADA Analyst III, Product Engineering & Delivery Services (ISD)
DHS Certified Trusted Tester (TTV5)
[IAAP WAS circular badge and horizontal name logo for International Association of Accessibility Professionals (IAAP) Web Accessibility Specialist (WAS) credential. To the left is a dark blue circle with three lines of centered white text that read: IAAP Certified WAS. There is a smaller light blue circle that surrounds the dark blue inner circle that designates the WAS credential color scheme. To the right, two lines of dark blue text. Top text reads Web Accessibility Specialist, second line reads International Association of Accessibility Professionals.]<https://www.accessibilityassociation.org/s/wascertification>
Navy Federal Credit Union, 820 Follin Lane, Vienna VA 22180
W: 571-391-0356 | C: 206-778-1882

[Navy Federal Credit Union. Our members are the mission.]

From: jake abma <jake.abma@gmail.com>
Sent: Friday, March 4, 2022 1:28 PM
To: Bradley-Montgomery, Rachael <rmontgomery@loc.gov>
Cc: public-silver@w3.org; w3c-wai-gl@w3.org
Subject: [External] Re: [Protocols] Minutes for March 4th, 2022

Please note the approach from Dutch government as mailed before, this was mentioned / used as a proposed starting point for "Possible ways to evaluate whether a protocol was done" derived from https://www.digitoegankelijk.nl/toegankelijkheidsverklaring/over-de-verklaring<https://secure-web.cisco.com/1bksw5hQXg9PYfUbtvrgUY0SeOxLJv2nVMRihu-4saz5ni8i0lcbUxbIx0qNe8zBLb__wriTFKMLyuVqvLwApBhCQ9K32hl0ct9yhLY2QFTReg-EjOBmimAuLYz9dBcM1qDLPIS11ZZEs8styO8bv8lciyhSRm9BasIzTJHzuJX_C2KgnLBTDMLD0D3WEd6RmXp3O_vCC_yoBeR_Q9g8GZPZFnXs5vgqSzDFWtIUsL1VU_ugA1wYqzbxsMsPz8rhgtD0Hn6szzml9SkCV8Gyz4b4xaQtdbT0RjRQXxZHngT5lHIb-zOFi6-tSU51R87DVcGdsMhzECW6zIhM4FMC8SdjbqRWqwqrKBgxPOzA2NwUT8YrV61Y7cetZ7Jega9rdoVRTl5fTHHcTJX-oMyA4nQ_mrABs78UMEsVrDEzpK4Yo0skZSyqRJngKEqsAXrwepPEc8zV9N8w4nf6pNXNzNA/https%3A%2F%2Fwww.digitoegankelijk.nl%2Ftoegankelijkheidsverklaring%2Fover-de-verklaring>

1. Previously (before 2020): ONLY 100% PASS / FAIL Compliancy Approach - there were only two statuses (WCAG driven)

2. Present Day: The current approach has FIVE compliance statuses !!!

A: Fully Compliant
B: Partially compliant (= in control statement)
              "agency has appointed concrete improvement measures READ: ASSERTION / FOLLOWING PROTOCOLS"
C: First measures taken
              "Agency has taken concrete improvement measures to get that picture. READ: ASSERTION / FOLLOWING PROTOCOLS"
D: Doesn't meet
             " Legal obligation prescribes agencies take the necessary measures
              Agency is urged to appoint concrete measures within a certain period of time, including planning. READ: ASSERTION / FOLLOWING PROTOCOLS"
E: No accessibility statement published

Op vr 4 mrt. 2022 om 16:11 schreef Bradley-Montgomery, Rachael <rmontgomery@loc.gov<mailto:rmontgomery@loc.gov>>:
The minutes from the protocols subgroup<https://www.w3.org/2022/03/04-wcag3-protocols-minutes.html> are available.

Summary:

  *   We will be going through an exercise for  the next few weeks to evaluate:

     *   How to evaluate whether the protocol was done
     *   How well the protocol was followed
     *   How to evaluate the quality of the results (if possible)

  *   We will be using the following (possible) protocols as examples to help with discussion:

     *   Plain Language, Visma UX, (BBC Gel A11y section if 3rd is needed)

  *   Possible ways to evaluate whether a protocol was done (Discussion still ongoing)

     *   Require the organization to publicly state:

        *   What protocol/part of protocol was done
        *   How the protocol was embedded in content or organization?
        *   How can the public see that the protocol was embedded?
        *   Date statement was made

  *   Key questions that need to be addressed later:

     *   Definition of a protocol?
     *   Is a protocol a document or part of a document?
     *   How will we handle overlap with WCAG? The overlap will shift

From: Chuck Adams <charles.adams@oracle.com<mailto:charles.adams@oracle.com>>
Date: Wednesday, March 2, 2022 at 1:42 PM
To: "public-silver@w3.org<mailto:public-silver@w3.org>" <public-silver@w3.org<mailto:public-silver@w3.org>>, "w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>" <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Subject: [Protocols] Agenda for March 4th, 2022
Resent-From: <w3c-wai-gl@w3.org<mailto:w3c-wai-gl@w3.org>>
Resent-Date: Wednesday, March 2, 2022 at 1:41 PM

Hi All,

The Protocols Subgroup will meet again this Friday, March 4th at 9:00 AM Boston Time (1400 UTC).

The Zoom teleconference data is provided at this link:

https://www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000<https://secure-web.cisco.com/1bBc-AeZLPPKOVvCExG0BbLyFYdagfVfi6hAuN7_FBdtZXh4Z-IDZQa11Uz5jnHurh-zQnEXFrU1PHmx0R7fK6XS-HoQLHIWyNdJXfTdBtc14_avFRLdaUczIaPPxQyTTNzok7P6cZuwwAP92oRfXqTX6DMFj_gnxyw6oMipmRtghaY8xXKoXcpIljDpqUy4soN38I3MvWyC5xosKGPRIJkKj3Js3DqWUeeq07_t5qtzGldc6Floe03VjydOTmwFybDeD-aw2EmyVKpI4A1kwGxDKgP3V1-_qZP1oxDokd0CrOKKRVZxN42-LOfekXZdm4uRmT1CRtyybc4sGJzRh7su1wxhrGnLgHAJr0AldhNVBQKloDmO9AOGHvxNAcRhTU0Ei4UB3aqXuz39egwtE0QxBpUuYchFWMZ45TAYabiOOBrAzZJFK6ib45TM0gi96JHEEoxvmuI7CV9EOno3gyQ/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.w3.org%2Fevents%2Fmeetings%2Fbfc72cd9-fdfc-4847-826a-01afb9e3f5e7%2F20211105T090000__%3B%21%21ACWV5N9M2RV99hQ%21ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34yacGIQO4g%24>

We will be on IRC using the W3C server at https://irc.w3.org<https://secure-web.cisco.com/1rI9UW3alosWZho6nVCm-E_dZkJ0BlrVmIyuglWkoUjfWBgRWm6nN35lmjZjFKf-INfhP42jq-fWcgnZi4RUDVZyTd_xV9MBOBw49_JX1_pme_MDZBZhjydUT4RaZ6RhKo0spYxpnqM39mlm1Lf_CG5Iu0Z9m9zjglM_9EkIVHROMeoDmfyAoOBMdLQRgmaYZ8IY_XcLmdZq4RO5Inf1p92eduUVe8o3W2peMTIyf-LVHNx3lugwhx5nr5v6d6gwGPBHRnntIKr1-FEnDe6A6OXihyqBL_vKW_vnHsF6e8sw-cLeuFeoZDWi7zaa5-jmzMTx3WcNZ_HZlwI2s6wh68dVVnRiVDH39gAi9MfLyJnYS7w9apy0GMFDaO9OHmJR0lTpKchQAqiUBvEj-mYvNwPpsMqsEU2IJFsprPrTeO2HSyb3JpM4ETuPgGnOxpzFjcFo2Lwo99TWCkAQDBMISHg/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Firc.w3.org%2F__%3B%21%21ACWV5N9M2RV99hQ%21ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ybOl3ZsYw%24>, in channel #wcag3-protocols

These and additional details of our work, including minutes, current, and archived draft documents are available on our subgroup wiki page here:

https://www.w3.org/WAI/GL/task-forces/silver/wiki/Protocols<https://secure-web.cisco.com/1tjim_fOcxm_qDZXrWClV8REdYA-GUtSZbZl_fRtKIrUFR4njzqvyNYSuAc7Xt_z9cn5ms3Sjt80esOl0OKBBv0TXcn8KiNRzh1hRVN1_SUm2gAYvqsZRug-VWDmK9K_UdFhBk1M6LXyskr-8fKqgelQVFptVtkudhNLxizoUUtmhcEQcQJVHmE11FQSn1eUiy1ioZ__ISn5ZXS7eK4HK2twbyRpxCrsizjlg9l24mGO9WRpi6ku0TJgOU5hMfwZE_bWRRevB7sZoeCoEacO8oje3QzDpoHtjX0mDZs0-9qd7aTJL3eNpmIgHu1w0dysTvGbfBLY2tufQdmh2EgSEmt_nGS2DzGikOxxSJlSJqPsePi4-ngHgiVZfj-qZuAIaBVT11nYM70RJrUhEts5Xfk9C_96nnhG_Ic-lgRU3mVmcBZ0OULXfHqr0qJrd3pNwnkJm25EaJUz0DcCM-c9wFA/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.w3.org%2FWAI%2FGL%2Ftask-forces%2Fsilver%2Fwiki%2FProtocols__%3B%21%21ACWV5N9M2RV99hQ%21ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ya-s3KL6w%24>

*** Agenda ***
agenda+ Develop a way for a lay-person to assess whether a protocol was followed


     *   Pick 2-3 things that are likely protocols (Plainlanguage.gov, BBC style guidelines, ?)
     *   Propose a way to evaluate (pass/fail):

                                                               i.      Whether the protocol was done

                                                             ii.      How well the protocol was followed

                                                           iii.      The quality of the results
Regards,
Charles Adams