Re: What if Silver didn't have levels?

I think some of the numbers here aren’t quite right.

The WebAIM Million<https://webaim.org/projects/million/> report found 97.8% had “detectable WCAG 2 failures”<https://webaim.org/projects/million/#wcag> on ONLY the home page of the 1M “most popular sites”,  (so rounds to 2%, not 3%), which they then go on to qualify:

“These are only automatically detectable errors that align with WCAG conformance failures with a high level of reliability. Because automatically detectable errors constitute a small portion of all possible WCAG failures, this means that the actual WCAG 2 A/AA conformance level for the home pages for the most commonly accessed web sites is very low, perhaps below 1%.”

So a better statement is that <= 1% of the HOME pages “conform to WCAG 2”.


Peter
--
Peter Korn | Director, Accessibility | Amazon Lab126
pkorn@amazon.com

From: John Foliot <john.foliot@deque.com>
Date: Monday, October 21, 2019 at 4:27 PM
To: Charles Adams <charles.adams@oracle.com>
Cc: "lwatson@tetralogical.com" <lwatson@tetralogical.com>, Matt King <a11ythinker@gmail.com>, Thomas Logan <thomas@equalentry.com>, Shawn Lauriat <lauriat@google.com>, Detlev Fischer <detlev.fischer@testkreis.de>, "Abma, J.D. (Jake)" <Jake.Abma@ing.com>, "public-silver@w3.org" <public-silver@w3.org>
Subject: Re: What if Silver didn't have levels?
Resent-From: <public-silver@w3.org>
Resent-Date: Monday, October 21, 2019 at 4:26 PM

> WebAIM found that 97% of the million most popular websites on the web have Level A and AA fails.

So, in other words, 3% met our current definition of conformant, which, when expressed as a percentile equals "100%" WCAG 2.0 conformance, which according to Léonie should also be a score we consider equal to useable.

But of the remaining 97% of those sites, how many are truly horrible? How many are bad, or fair, or not too bad? If one site of those 97% failed due to a color contrast issue (alone), then I suspect that the offending site would remain a functional 100% usable for screen reader users, but would still end up on Jared's fail list, and would still  be failing some users.

I will continue  to suggest then that usable cannot equal 100%, because Jared has proven beyond debate that in a pass/fail world, most sites fail.

I wonder aloud then, of that 97% collection of sites evaluated and deemed "failures", how many were usable, even if not fully conformant? How many were, in aggregate, mighty well done, except for maybe 2 missing alt texts (from 500 images in total) and had poor focus indication in the footer (alone)? If usable isn't the same as perfect (100%), then what should usable equal? Or, if "flawed but usable" = 100%, then what does perfect equal? 150%? (something of an oxymoron)

There isn't a tonne of light between Léonie and I in our goals, I simply would like to see a realist number, something less than perfect, be a minimal score, below which we all can collectively agree "not good enough".

But at the same time regretably, sometimes "good enough" (or "Good enough for government work" as a good friend often says <grin>) has to be an answer; that in a real and flawed world we must accept a little water with our wine, because the alternative is a list of 1 million sites that, according to our current conformance model, suggests 97% aren't "accessible", even though for many users - including many users with differing disabilities - they'll still likely find quite usable, even while still potentially looking up and saying, "Really good, but..."

(I'll also suggest that "useable" as a means of measurement is subjective, which also concerns me...)

JF

(Sent from my mobile, apologies for any spelling mistakes)

On Mon, Oct 21, 2019, 5:14 PM Chuck Adams <charles.adams@oracle.com<mailto:charles.adams@oracle.com>> wrote:
I will take a crack at an explanation, though hopefully it doesn't trample on Matt's posts or points.

W3C (us) created a standard system.  WCAG 2.0.  Was it defect free?  Did it cover all user needs of every ability?  No.  As you mention, it has failures (you mention one, but there are other defects which have been acknowledged).

AGWG (us) worked on and released WCAG 2.1.  It also wasn't defect free, and it also didn't cover all user needs of every ability.

We are working on WCAG 2.2.  I posit that it won't be defect free, and it won't cover all user needs of every ability.

Silver will hopefully be a paradigm shift.  We will hopefully learn and apply lessons from the past (possibly even the one you mention).  We are discussing that it can scale, and that it can be extensible, and that we can update it as it evolves.  This indicates to me that we anticipate it will not be defect free and that it will not offer all the guidance needed to cover all users of every ability.  Anticipating this, we are trying to make it updatable and extensible.  I don't expect our first release of Silver to be the last set of guidance ever needed, and that it will address everyone's needs.

If we can craft a sets of guidance that will be defect free and will cover all user needs of every ability, then I concede that we can make the same demands of software.  If we aren't able to craft such guidance, then I believe that we ought not make higher demands from complex software.

Regards,
Chuck

-----Original Message-----
From: Léonie Watson <lwatson@tetralogical.com<mailto:lwatson@tetralogical.com>>
Sent: Monday, October 21, 2019 2:27 PM
To: Matt King <a11ythinker@gmail.com<mailto:a11ythinker@gmail.com>>; Thomas Logan <thomas@equalentry.com<mailto:thomas@equalentry.com>>; John Foliot <john.foliot@deque.com<mailto:john.foliot@deque.com>>
Cc: Shawn Lauriat <lauriat@google.com<mailto:lauriat@google.com>>; Detlev Fischer <detlev.fischer@testkreis.de<mailto:detlev.fischer@testkreis.de>>; Abma, J.D. (Jake) <Jake.Abma@ing.com<mailto:Jake.Abma@ing.com>>; public-silver@w3.org<mailto:public-silver@w3.org>
Subject: Re: What if Silver didn't have levels?

On 21/10/2019 17:09, Matt King wrote:
Matt wrote:
>[...]

> On the other hand, I am pretty certain hat a standards system that
> does not enable people to draw lines through the gray is certain to fail.

Can you explain why?

We have substantial evidence that confirms the level approach has failed. We know nobody bothers with Level AAA (not even people committed to accessibility really), and WebAIM found that 97% of the million most popular websites on the web have Level A and AA fails.

--
Director @TetraLogical