Re: What if Silver didn't have levels?

Léonie writes:

> I think it's worth looking at the inversion model Alastair mentioned
though. Instead of starting at 0% and working your way up, you start at
100% and try not to work your way down.

> To continue with my earlier example, this would change to the following:
There is a requirement to provide captions for multimedia content, if you
don't you deduct 1%, if there is no ability to adjust the location
of the captions, you deduct 2%.

But, you still haven't solved the real problem, which (if I may) is
essentially, "if we cannot achieve perfect, how good is good enough?"

In Leonie's example (and NOT discussing the specific issue), let's say that
the ONLY thing not in compliance is the missing, movable captions, so OK,
minus 2%. You now have a 98% score. (And why 1% or 2%? Why not 5% or 10%?
What is a "percentage point" worth? And why?)

Now, I personally think that achieving a 98% "Success score" is pretty
darned awesome, even today, but at what point is "awesome" replaced by
"pretty danged good" or perhaps "good enough" or "not very good but
workable" or "that's horrible"? (I note that when speaking with colleagues
with disabilities about specific sites, they frequently use language like
"it's not bad" or "it's pretty good" or "it's really painful to use with my
AT", so the idea that good enough exists in the wild is borne-out by those
statements). And while Shawn notes that "60%" is today a strawman number
(with others suggesting that number may be too low), in the end whether
it's additive or subtractive, the scoring mechanism *also* needs enough
flexibility to accommodate more requirements as they emerge for new
technologies (*), and/or allows for the "subtraction" of percentile scoring
requirements when they are not applicable.

We also have a requirement to ensure that we don't "stack" points in favor
of one user-group, at the expense of a different user-group.

What if, instead of missing captions, it was a site missing multiple
textual alternatives? What does "missing text alternatives" count for? 1%?
2%? 5%? More?  Is that score based upon each image on a page, a
cumulative score of all images on the page, a grand total of all images on
all pages, or a representative sampling? Text heavy (or text-only) sites
will likely get a positive bump there, whereas image-rich pages/sites may
take a biased hit if we base it simply on number of images on a page or
site alone. Yet, depending on each actual image, the severity and impact on
the end-user of a missing text alternative could conceivably range from
merely annoying to out-right dangerous.

We also know that for issues related to cognitive accessibility, measuring
"success" or "failure" is going to involve more needs-based and role-based
evaluations (which will take longer to perform, are more open to subjective
interpretation, and likely will not happen as frequently as more basic,
mechanical testing). I've previously suggested the idea of "time-stamping"
those forms of test results, with a mechanism that stale-dates the score
over time (based on the assertion that often what was tested in January
likely isn't still there in October of the same year, never mind from 2
years ago... This is the concern/problem I believe Peter Korn is focusing
on right now but in mirror relief - how do we measure a site that is
constantly adding new content?)

Earlier,  Léonie wrote:

> The incentive thing still works, it's just that you strive to improve
your score instead of to get from "good" to "best".
In legal cases the determination can be made on the score


From a "legal" perspective, that might work as a reactive approach, but
most entities today are looking for proactive approaches to avoid any legal
complications. From that perspective, site owners *DO* need a Good
(enough), Better, Best, (and Awesome) set of targets to strive towards. We
cannot somehow be asking sites to boil the ocean right out of the gate -
they will need achievable milestones, of which Good, Better, Best is as
good, or benign, or bad as any other scheme.

I believe then that we need to start with the realistic assumption that
nobody will ever achieve "Perfect", and then define what a "MVP" (Minimum
Viable Product) of "accessibility success" would look like. Failing to meet
that MVP milestone (however it is scored) = Failure, and meeting or
exceeding that same MVP milestone = "some degree of" Success; recognizing
that it's still not "perfect", but that it isn't "horrible" either.

Failing to provide some kind of "sliding scale for measurement" (aka the
"levels" in this thread) simply leaves us where we are today: the flawed
"Perfect or  Complete Failure" conformance model we have with WCAG 2.x.  I
think that it is also important to remember that "conformance" levels are
significantly more important to regulators than to end users, and I've had
enough time and experience to observe that even determining "Good Enough"
for end-users is individualistic in nature.

JF (returning to lurk mode)


(* as an additional question... what of emergent technologies where *all*
of WCAG as we know it today is not applicable, but some is, along with some
"new" requirements specific to that new technology? I'm thinking now of AR
/ VR / XR (aka "Immersive" web) - how do we score that when, for example,
we may not have a technique to "declare the language of the page", or
provide a "mechanism to bypass blocks" in Virtual Reality?)

On Fri, Oct 18, 2019 at 11:24 AM Léonie Watson <lwatson@tetralogical.com>
wrote:

> I don't think that setting a level somewhere on the scale is a good
> idea. As soon as we do that, we validate the question "what is the least
> I have to do here?", and as soon as that happens we've accepted that
> some people will be disadvantaged. In other words we create a class system.
>
> If we instead set the "level" at 100%, we solve this problem. It also
> makes the definition of "accessible" a lot easier for people to
> understand - whether you're in court or implementing a new interface,
> people understand that 100% is the most you can achieve.
>
> I think this single "level" becomes even more important if we think
> about bonuses.
>
> A problem with bonuses is that if they're awarded for meeting additional
> requirements, it effectively reintroduces the levels problem. We know
> this from all the Level AAA requirements that are never met.
>
> But if we offer bonuses for meeting requirements in ways that are known
> to be superior, or in ways that offer a better UX, I think we can
> protect against this risk too.
>
> For example, there is a requirement to provide captions for multimedia
> content, and it's worth 1%. If you make it possible to adjust the
> location of the captions, that gets you a bonus 1%.
>
> Please don't get into the weeds of discussing this example. It almost
> certainly won't stand up to the tiniest bit of scrutiny. It's just meant
> to be an example to illustrate the idea.
>
> I'm also wary of the idea of "grandfathering" WCAG into the Silver
> model. As soon as we say that X point on the Silver scale is the same as
> Level AA, we implicitly reintroduce the class system I think is such a
> terrible idea. We transitioned from WCAG 1.0 to 2.0 without any real
> mapping between the two, and so I have no reason to think the same thing
> can't happen when we transition from WCAG2.x to Silver.
>
> I think it's worth looking at the inversion model Alastair mentioned
> though. Instead of starting at 0% and working your way up, you start at
> 100% and try not to work your way down.
>
> To continue with my earlier example, this would change to the following:
>
> There is a requirement to provide captions for multimedia content, if
> you don't you deduct 1%, if there is no ability to adjust the location
> of the captions, you deduct 2%.
>
>
> Léonie.
> On 18/10/2019 15:48, Shawn Lauriat wrote:
> > That 60% number came from Alastair writing "For sake of argument, that
> > level could be 60%…" so we definitely shouldn't take that as a solid
> > proposal that WCAG 2.x AA means 60% passing in Silver. Having a number
> > just made it so that we could work through the other aspects of this
> > conversation, which I've personally found really helpful!
> >
> > Detlev & Makoto, the examples you each gave for ways of adding to WCAG's
> > conformance in order to better express granularity of accessibility
> > beyond yes/no and A/AA/AAA seem particularly helpful to reference and
> > think about.
> >
> > Thanks,
> >
> > Shawn
> >
> > On Fri, Oct 18, 2019 at 8:08 AM Detlev Fischer
> > <detlev.fischer@testkreis.de <mailto:detlev.fischer@testkreis.de>>
> wrote:
> >
> >     Ok, I see what you mean!
> >
> >     Am 18.10.2019 um 14:00 schrieb Abma, J.D. (Jake):
> >>     Not sure if I understand you well, but 100% now will be 60%
> tomorrow.
> >>
> >>     So the 50 SC now will be extended to 80 in near future and when
> >>     60% of the 80 are passed (about 48/50/what ever we have) you
> >>     barely pass.
> >>
> >>     When more SC/methods are introduced you will need to catch up
> >>     because your 60% will drop a bit, encouraging to improve
> >>     constantly in future releases of the standard.
> >>
>  ------------------------------------------------------------------------
> >>     *From:* Detlev Fischer <detlev.fischer@testkreis.de>
> >>     <mailto:detlev.fischer@testkreis.de>
> >>     *Sent:* Friday, October 18, 2019 1:53 PM
> >>     *To:* Abma, J.D. (Jake) <Jake.Abma@ing.com>
> >>     <mailto:Jake.Abma@ing.com>; public-silver@w3.org
> >>     <mailto:public-silver@w3.org> <public-silver@w3.org>
> >>     <mailto:public-silver@w3.org>
> >>     *Subject:* Re: What if Silver didn't have levels?
> >>     Up to now there was even talk of 'grandfathering'? WCAG 2.1
> >>     coformant results so sites conforming to current level AA would
> >>     meet the basic level, bronze (not sure if that is still the
> >>     favoured scheme though). That would be meeting a 100% old SCs (I
> >>     guess with minor tolerances) as a baseline requirement, if I
> >>     understand correctly. Lowering that to 60% (whatever the
> >>     rearrangements and additions coming with Silver) still feels like
> >>     a very significant lowering of requirements. Personally I would
> >>     not want to support that.
> >>
> >>     Am 18.10.2019 um 13:47 schrieb Abma, J.D. (Jake):
> >>>     (this makes the test and result a bit more complicated of course,
> >>>     but most people already understand a score like that)
> >>>
>  ------------------------------------------------------------------------
> >>>     *From:* Abma, J.D. (Jake) <Jake.Abma@ing.com>
> >>>     <mailto:Jake.Abma@ing.com>
> >>>     *Sent:* Friday, October 18, 2019 1:45 PM
> >>>     *To:* Detlev Fischer <detlev.fischer@testkreis.de>
> >>>     <mailto:detlev.fischer@testkreis.de>; public-silver@w3.org
> >>>     <mailto:public-silver@w3.org> <public-silver@w3.org>
> >>>     <mailto:public-silver@w3.org>
> >>>     *Subject:* Re: What if Silver didn't have levels?
> >>>     Detlev,
> >>>
> >>>     True, but this may depend on where we set the 60% baseline.
> >>>     If double A (AA) is the baseline / 60% and some AAA are part of
> >>>     the default scoring system, than you can increase the score more
> >>>     easily.
> >>>
> >>>     Like touch targets and heading structure NOT AAA like but part of
> >>>     the defaults (and more to come...)
> >>>     To see how this works it might be good to have concrete practical
> >>>     examples to talk about.
> >>>
> >>>     The friction of focussing to much on 1 disability can only be
> >>>     solved by breaking the SC/guidelines/methods apart and judge the
> >>>     group to then demand a baseline score 'per group'.
> >>>     (just like we have in our schools, you must have a 60% average
> >>>     for math, language, history, physics etc in order to graduate)
> >>>
> >>>     Cheers!
> >>>
> >>>
>  ------------------------------------------------------------------------
> >>>     *From:* Detlev Fischer <detlev.fischer@testkreis.de>
> >>>     <mailto:detlev.fischer@testkreis.de>
> >>>     *Sent:* Friday, October 18, 2019 1:35 PM
> >>>     *To:* Abma, J.D. (Jake) <Jake.Abma@ing.com>
> >>>     <mailto:Jake.Abma@ing.com>; public-silver@w3.org
> >>>     <mailto:public-silver@w3.org> <public-silver@w3.org>
> >>>     <mailto:public-silver@w3.org>
> >>>     *Subject:* Re: What if Silver didn't have levels?
> >>>     Jake,
> >>>     I agree withj sepatrate layers, "basic" and "extra on top" and
> >>>     critical issues
> >>>
> >>>     However (and I may be repeating myself) a result of 60%, in our
> >>>     old scoring scheme, would describe REALLY bad sites. This is
> >>>     because a good number of SCs rarely see fails (think of 1.3.3.
> >>>     Sensory Characteristics, 1.3.4 Orientation, 1.4.5 Images of Text,
> >>>     2.3.1 Three flashes, 2.4.5 Multiple ways, 2.5.4 Motion
> >>>     Activation, 3.1.1 Language of Page, 3.2.1 On Focus, 3.2.2 On
> >>>     Input). So if bottom line of 60% means that a whooping 40% of SCs
> >>>     can have defects (the queston question is of cause, *how* bad),
> >>>     that baseline would allow for pretty dismal stuff. So I fear this
> >>>     would act as a disincentive for those who are happy to just
> >>>     scrape through with minimal conformance, and really lower the bar
> >>>     established in WCAG 2.X.
> >>>
> >>>     I realise the structure of Silver will be different but the
> >>>     testable issues will be largely similar, in whatever arrangement
> >>>     nameing or granularity.
> >>>
> >>>     Am 18.10.2019 um 13:09 schrieb Abma, J.D. (Jake):
> >>>>
> >>>>     Think we need some leeway here and there, but keep it very
> >>>>     simple, like:
> >>>>
> >>>>     BASICS:
> >>>>
> >>>>       * Seems like a simple / plain, 100 point with 60% baseline
> >>>>         system, might do the trick to keep it clean.
> >>>>
> >>>>     EXTRA ON TOP:
> >>>>
> >>>>       * For the AAA like-ish (best practices) you could add bonus
> >>>>         points to it (clear but limited set!)
> >>>>       * And have a sort of severe/ critical issues /
> >>>>         Non-Interference musts (subtract points?! clear but limited
> >>>>         set!)
> >>>>
> >>>>     Result like:
> >>>>
> >>>>       * Basic score: 73%
> >>>>       * Bonus points: 20/100 (namely the following methods: ... and
> ...)
> >>>>       * Critical issues: NONE
> >>>>
> >>>>     Cheers!
> >>>>     Jake
> >>>>
>  ------------------------------------------------------------------------
> >>>>     *From:* Detlev Fischer <detlev.fischer@testkreis.de>
> >>>>     <mailto:detlev.fischer@testkreis.de>
> >>>>     *Sent:* Wednesday, October 16, 2019 1:54 PM
> >>>>     *To:* public-silver@w3.org <mailto:public-silver@w3.org>
> >>>>     <public-silver@w3.org> <mailto:public-silver@w3.org>
> >>>>     *Subject:* Re: What if Silver didn't have levels?
> >>>>     One problem I see is that 60% (even 75%) sounds like there would
> >>>>     already
> >>>>     be MANY possibly grave accessibility issues -- when you add AAA
> >>>>     criteria
> >>>>     and best practices to such a site, these won't go away. I would
> >>>>     imagine
> >>>>     some sort of minimum conformance baseline should be set
> >>>>     independent of
> >>>>     extra things - aspects now at AAA - and I find it dubious that
> >>>>     you would
> >>>>     be able to improve one overallscore by these (possibly
> >>>>     non-essential)
> >>>>     additions.
> >>>>
> >>>>     Am 16.10.2019 um 13:10 schrieb Alastair Campbell:
> >>>>     > If the methods that are currently at WCAG 2.1 AA got you to
> >>>>     60% (for
> >>>>     > example), you’d need to do things at the AAA level & best
> >>>>     practices to
> >>>>     > score higher.
> >>>>
> >>>>     --
> >>>>     Detlev Fischer
> >>>>     Testkreis
> >>>>     Werderstr. 34, 20144 Hamburg
> >>>>
> >>>>     Mobil +49 (0)157 57 57 57 45
> >>>>
> >>>>     http://www.testkreis.de
> >>>>     Beratung, Tests und Schulungen für barrierefreie Websites
> >>>>
> >>>>
> >>>>     -----------------------------------------------------------------
> >>>>     ATTENTION:
> >>>>     The information in this e-mail is confidential and only meant for
> the intended recipient. If you are not the intended recipient, don't use or
> disclose it in any way. Please let the sender know and delete the message
> immediately.
> >>>>     -----------------------------------------------------------------
> >>>
> >>>     --
> >>>     Detlev Fischer
> >>>     Testkreis
> >>>     Werderstr. 34, 20144 Hamburg
> >>>
> >>>     Mobil +49 (0)157 57 57 57 45
> >>>
> >>>     http://www.testkreis.de
> >>>     Beratung, Tests und Schulungen für barrierefreie Websites
> >>>     -----------------------------------------------------------------
> >>>     ATTENTION:
> >>>     The information in this e-mail is confidential and only meant for
> the intended recipient. If you are not the intended recipient, don't use or
> disclose it in any way. Please let the sender know and delete the message
> immediately.
> >>>     -----------------------------------------------------------------
> >>
> >>     --
> >>     Detlev Fischer
> >>     Testkreis
> >>     Werderstr. 34, 20144 Hamburg
> >>
> >>     Mobil +49 (0)157 57 57 57 45
> >>
> >>     http://www.testkreis.de
> >>     Beratung, Tests und Schulungen für barrierefreie Websites
> >>     -----------------------------------------------------------------
> >>     ATTENTION:
> >>     The information in this e-mail is confidential and only meant for
> the intended recipient. If you are not the intended recipient, don't use or
> disclose it in any way. Please let the sender know and delete the message
> immediately.
> >>     -----------------------------------------------------------------
> >
> >     --
> >     Detlev Fischer
> >     Testkreis
> >     Werderstr. 34, 20144 Hamburg
> >
> >     Mobil +49 (0)157 57 57 57 45
> >
> >     http://www.testkreis.de
> >     Beratung, Tests und Schulungen für barrierefreie Websites
> >
>
> --
> Director @TetraLogical
>
>

-- 
*​John Foliot* | Principal Accessibility Strategist | W3C AC Representative
Deque Systems - Accessibility for Good
deque.com