- From: John Foliot <john@foliot.ca>
- Date: Fri, 4 Mar 2022 16:21:44 -0500
- To: "Bradley-Montgomery, Rachael" <rmontgomery@loc.gov>
- Cc: "public-silver@w3.org" <public-silver@w3.org>, "w3c-wai-gl@w3.org" <w3c-wai-gl@w3.org>
- Message-ID: <CAFmg2sXTw37UCjG-8uCbmRwdC4AHeE+1qxs1pL+BGxKZNjAdHw@mail.gmail.com>
Thank you for posting the minutes of today's call. As I read them, I noted the following: <Rachael> ...Jeanne spoke 3-4 years ago with lawyers there was the possibility to add more subjectiveness. I'm not sure that's true or not. Its about measuring the effort. While I can respect that perspective, I must also strongly disagree. Protocols are to address needs/requirements that *cannot be measured*, so respectfully it's not about measuring the effort (and how would you do that?), it is about *soliciting the commitment*, and having that commitment *publicly documented* (the protocol and the 'promise'). - "...the point of protocols is to achieve outcomes that we don't know how to measure." - "...protocols are about procedures you use to meet an outcome that is not easily measurable" (source: https://www.w3.org/WAI/GL/task-forces/silver/wiki/Protocols#14_January_2022 ) Again, using the example of Plain Language (as outlined by plainlanguage.gov) - the 'effort' to ensure plain language will also be directly correlated to the content itself: some content will be harder to express in plain language than other content, so the effort will be *mostly* based on the content. In that scenario, things that 'could be measured' that I can think of include the number of hours it took, impact on staffing needs (it took 3 people 2 days, versus it took 1 person half-an-hour - but that does not account for experience or expertise, both factors in determining "how long"), contrasting time to author/remediate against volume of content (paragraphs, pages, chapters...) - none of those metrics relate to the final output however, which is still intended to be arriving at *content written in plain language* (audience reading level, comfort in reading the content, etc.). Measuring the effort does not correlate to the final outcome in any meaningful way that I can think of. *************** I also noted the discussion on what makes a protocol and who decides. In my initial proposal, I had suggested two types of protocols, with differing levels of "reward" (points toward a final score that leads to Bronze/Silver/Gold). *Vetted and Approved:* I had suggested that the first 'class' (type) of protocol would be a protocol specifically called out (vetted) by the Working Group. In that case, subject matter experts within our group can determine both the suitability and impact that specific protocols would bring to the overall outcomes, and we could assign those specific protocols 'points' based on our judgement. This kind of protocol could come directly from the W3C/WAI itself, or our own experts could 'validate' other protocols authored by governments, companies, and other entities. (In my proposal I had offered as a straw-man suggestion Adobe's User & Usability Testing Questions - https://xd.adobe.com/ideas/process/user-testing/usability-testing-questions-tips-examples/ ) For example, "Making Content Usable for...COGA <https://www.w3.org/TR/coga-usable/>"(*) is another document that I've frequently suggested meets *my* mental model of a Protocol: like the US Plain Language content currently being used for evaluation, "Making Content..." has 8 specific 'Outcomes' that cannot be measured granularly, however because of how that document is written, it clearly provides both content creators and content evaluators with enough data to make informed decisions (explanations, user stories, etc.) - both in the authoring phase as well as in the 'evaluation' phase. The document establishes parameters and expected outcomes to "look for", but falls short of stating, "you must do this, or you must do that". In this case, because it is a document (protocol) that comes from within, I would suggest we assign it a 'higher' value when contributing to a final score. (* My proposal for "Protocols" was very much driven by me originally and directly asking myself "How can we include "Making Content..." into WCAG 3?" I believe it is both an excellent resource that addresses a number of real gaps and problems today, and it clearly - at least to me - defines the non-measurable requirements that the COGA group determined were required 'at scale'. Given that both plainlanguage.gov and "Making Content..." have been authored with a similar structure, they may also be the basis of a 'protocol template' we could look at) *Internal or Company-Authored:* This may be a bit trickier (controversial?), but I believe it will be an important part of the larger effort. If our WG clearly defines the structure of what we expect to see in a protocol (minimum requirements), we could then potentially let content creators author their own protocols - especially when it comes to specialized content (strawman: META publishes a protocol for XR environments that addresses accessibility needs. Allowing custom protocols encourages new technologies to also document accessibility considerations at a scale faster/larger than we can handle here within the W3C - and so should be actively encouraged in my mind). Of course, custom protocols would still need to be directly addressing the needs of users with disabilities (and we could potentially even mandate proof of direct consultation with impacted user-groups as part of the requirements for such custom protocols). In that scenario, to be considered a 'valid' protocol, at a minimum I would expect that the document/protocol itself be *Publicly available* (no paywalls or intranet-only documents) so that any evaluator could easily reference that 'promise' to determine if it has been applied to the content. In my previous email I had proposed one method of how to provide that reference (i.e. it's a public URL) into an assertion statement. In terms of 'scoring' however, those protocols would be worth "less", primarily because they have not been directly vetted by the W3C WG. (I could also envision a mechanism where 3rd parties could propose a protocol to our group for consideration - if we agreed it was hitting the mark, we could then elevate that protocol to the "vetted list" - allowing for extensibility. In my initial presentation, I had referenced *Moodle Usability testing protocol* [ https://docs.moodle.org/dev/Usability_testing_protocol] as a potential example of this; the group seems to have also landed on BBC Gel, which would be a similar document that today is unvetted, but potentially could 'graduate' to our list of vetted protocols - ditto the Adobe doc referenced earlier). Respectfully, JF On Fri, Mar 4, 2022 at 10:09 AM Bradley-Montgomery, Rachael < rmontgomery@loc.gov> wrote: > The minutes from the protocols subgroup > <https://www.w3.org/2022/03/04-wcag3-protocols-minutes.html> are > available. > > > > Summary: > > - We will be going through an exercise for the next few weeks to > evaluate: > - How to evaluate whether the protocol was done > - How well the protocol was followed > - How to evaluate the quality of the results (if possible) > - We will be using the following (possible) protocols as examples to > help with discussion: > - Plain Language, Visma UX, (BBC Gel A11y section if 3rd is needed) > - Possible ways to evaluate whether a protocol was done (Discussion > still ongoing) > - Require the organization to publicly state: > - What protocol/part of protocol was done > - How the protocol was embedded in content or organization? > - How can the public see that the protocol was embedded? > - Date statement was made > - Key questions that need to be addressed later: > - Definition of a protocol? > - Is a protocol a document or part of a document? > - How will we handle overlap with WCAG? The overlap will shift > > > > *From: *Chuck Adams <charles.adams@oracle.com> > *Date: *Wednesday, March 2, 2022 at 1:42 PM > *To: *"public-silver@w3.org" <public-silver@w3.org>, "w3c-wai-gl@w3.org" < > w3c-wai-gl@w3.org> > *Subject: *[Protocols] Agenda for March 4th, 2022 > *Resent-From: *<w3c-wai-gl@w3.org> > *Resent-Date: *Wednesday, March 2, 2022 at 1:41 PM > > > > Hi All, > > > > The Protocols Subgroup will meet again this Friday, March 4th at 9:00 AM > Boston Time (1400 UTC). > > > The Zoom teleconference data is provided at this link: > > > https://www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000 > <https://urldefense.com/v3/__https:/www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000__;!!ACWV5N9M2RV99hQ!ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34yacGIQO4g$> > > We will be on IRC using the W3C server at https://irc.w3.org > <https://urldefense.com/v3/__https:/irc.w3.org/__;!!ACWV5N9M2RV99hQ!ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ybOl3ZsYw$>, > in channel *#wcag3-protocols* > > These and additional details of our work, including minutes, current, > and archived draft documents are available on our subgroup wiki page here: > > https://www.w3.org/WAI/GL/task-forces/silver/wiki/Protocols > <https://urldefense.com/v3/__https:/www.w3.org/WAI/GL/task-forces/silver/wiki/Protocols__;!!ACWV5N9M2RV99hQ!ZvVx1wh89EAXhBiorHpgvdpQRlEtQPxaEsJbJ7_Q3MrxtnQGs5lwbIC34ya-s3KL6w$> > > *** Agenda *** > > agenda+ Develop a way for a lay-person to assess whether a protocol was > followed > > > > 1. Pick 2-3 things that are likely protocols (Plainlanguage.gov, BBC > style guidelines, ?) > 2. Propose a way to evaluate (pass/fail): > > i. Whether > the protocol was done > > ii. How > well the protocol was followed > > iii. The > quality of the results > > Regards, > > Charles Adams > -- *John Foliot* | Senior Industry Specialist, Digital Accessibility | W3C Accessibility Standards Contributor | "I made this so long because I did not have time to make it shorter." - Pascal "links go places, buttons do things"
Received on Friday, 4 March 2022 21:22:17 UTC