Re: Costs of testing with Silver

Hi Wilco,

I'm certainly not suggesting that you wish to "ignore the needs of certain
people", only that a simplified (no cost/low cost) testing strategy may
cause a false sense of "success", or somehow diminish the need for robust
testing, which ultimately does leave more users behind.

It's easy to argue that Site A at "70% accessible" is better than Site B at
"35% accessible", and in a positive light that equals one site that is
twice as accessible as another, but in either instance the sites are still
inaccessible to between 30% and 65% of their potential users. I struggle
with that, and I further struggle with us saying that 70% is good enough
for smaller or "simpler" sites and shops that cannot (or will not) pony up
to do the proper testing. To my thinking, half-baked testing is as lousy as
no testing to the end user.

Looking at your bullet points:

> - Shift responsibilities over to UA and AT vendors, instead of content
authors. For example have ATs solve the issue of single key shortcuts, and
require browsers to always have a visible tab focus for keyboard users.

I fully support that idea, however *HOW* do we accomplish that? The W3C
does not have a mandate or mechanism to force anyone to do anything, and so
while the idea is right in principle, implementation remains, well, where
we are today.

Have I given up hope? No, but I am also pragmatic in thinking that vendors
alone won't solve some of the problems, especially as the impacted
user-group decreases in volume. Today, all browsers *DO* have native
visible tab focus indication (with at least a few using an indication that
fails contrast requirements), but thanks to author control, that visible
tab focus can be "turned off", and I'll bet a year's salary that no browser
will remove that ability any time soon.

Additionally, not all accessibility issues are hardware or software-based
issues: often it is content being created by the site owner that is
introducing the issues. We've all seen this multiple times, especially when
authors are using CMS tools, where the templates may be "accessible", and
yet the content author writes about clicking on the round red button on the
right - another "easy" accessibility problem to point out, but I'm not sure
that the kinds of no-cost/low-cost testing strategies such as you are
proposing could capture that problem. (Given internationalization concerns,
I struggle to think of any automated test that could be applied directly to
authored content)


> - Use more requirements such as parsing that look for bad practices known
to lead to inaccessible content, instead of things that are directly
inaccessible. Those things tend to be much easier to test (think "don't
skip heading levels")


While I understand your point, the example you pointed out is only really
an issue to screen reader users, as sighted users generally don't see the
"level", but rather the outcome when that "level" has been styled. A
heuristic test like that is still important, don't get me wrong, but it is
but a sliver of the number of issues that can be found, and again relies on
"structure" as opposed to "content", which at least 10 of the current WCAG
2.0 SC are dependent on successfully meeting today.


> - Remove some of the complexities and exceptions in the entry level. For
example the "media alternative for text". Maybe we have a "simple sites"
version of Silver that doesn't have this stuff, but a "complex site" which
allows them, but adds in stuff that's rarely a problem on simple sites.


Again, I understand the larger point, but with that approach we have to
also declare, formally, the difference between a "simple" site and a
"complex" site.
Is a one page site that features a professor's video lecture at a
university "simple" or "complex"? Why?

That discussion could go either way: on the simple side it's a web page
with a single heading, a video rendering area, and a footer that says
"Copyright XYZ University". Yet because that single, simple page contains a
video, it has an additional slew of requirements that cannot be accurately
machine tested.

Moving beyond that, say that the single page I just outlined was one of a
total of 5 pages created by the Professor, and all of the 5 pages use the
same basic template (header, nav, main, footer). It would be very easy to
claim it's a "simple" site (because 80% of it is), but the remaining 20%
(the video page) is "complex". Now what?

If four of the 5 pages "pass", yet the video page is an abject failure, I
can see the site owner arguing "ya, but I got an 80% pass score" but the
impacted blind, deaf, or deaf/blind users are still left with...
potentially nothing. This is the concern - that, and defining what is
"simple" versus what is "complex", which is another subjective call at a
higher level.


> - Remove ambiguities and technology agnostic language from definitions.
Even if we do it just for HTML and SVG. If we can make it unambiguous,
that's even better because that means it's automatable.

+1 to removing ambiguity, although I am less sure about removing technology
agnostic language from the definitions or Success Criteria.
Since the beginning, WCAG 2.0 has mostly been about functional outcomes,
and not methods of meeting a specific SC (which is one of the reasons why
Techniques are both non-normative, and outside of the normative spec), and
so keeping the ideas open and broad, without specific techniques, is a goal
I would continue to support.


>

All of these things help reduce the costs of accessibility testing without
impacting PwD in the slightest.

I'm not sure I understand what you mean here: if the tests *aren't*
impacting PwD (in a positive way), why are we testing? To prove we did a
test? (That again sounds like check-box accessibility, WCAG 1.0-style)

Don't get me wrong, the more we can do to make 'testing' easier, faster,
cheaper, and more accurate is a goal I fully support. My concern arises
when we start to suggest that "accessibility testing" is but a button click
away, and that needing to *think* about accessibility is less important
than using patterns that match accessibility techniques. That's because, as
experienced professionals, we all know that half the time, the answer to
any given accessibility question usually starts out with, "That depends..."

I recognize that cost is a factor, I really do. But to my mind, quality
must supersede cost if we truly want to benefit the target audience of PwD.
Focusing on the lowest common denominator (make it cheap) also has a cost -
to the end user. It's not a trade-off I personally am comfortable with.

JF


On Wed, Sep 5, 2018 at 1:01 PM, Wilco Fiers <wilco.fiers@deque.com> wrote:

> Hi John, Charles,
> Neither of you have responded to the actual concerns that I've raised
> that, we should be concerned in making accessibility testing affordable for
> smaller organisations with no legal requirement to follow accessibility
> standards.
>
> I am not suggesting that we ignore the needs of PwD. Making this out as
> though I've suggested that we ignore the needs of certain people is
> misrepresenting my concern. There are a lot of ways we can approach this
> concern of cost that wouldn't impact PwD in the slightest. Just to list a
> few:
>
> - Shift responsibilities over to UA and AT vendors, instead of content
> authors. For example have ATs solve the issue of single key shortcuts, and
> require browsers to always have a visible tab focus for keyboard users.
> - Use more requirements such as parsing that look for bad practices known
> to lead to inaccessible content, instead of things that are directly
> inaccessible. Those things tend to be much easier to test (think "don't
> skip heading levels")
> - Remove some of the complexities and exceptions in the entry level. For
> example the "media alternative for text". Maybe we have a "simple sites"
> version of Silver that doesn't have this stuff, but a "complex site" which
> allows them, but adds in stuff that's rarely a problem on simple sites.
> - Remove ambiguities and technology agnostic language from definitions.
> Even if we do it just for HTML and SVG. If we can make it unambiguous,
> that's even better because that means it's automatable.
>
> All of these things help reduce the costs of accessibility testing without
> impacting PwD in the slightest. Now I'm not saying we shouldn't shift some
> things around. Perhaps we can pick apart existing requirements such as
> those for info and relationships and keep the essential part in the entry
> level, but move the less common / less burdensome issues to the "Silver /
> AA" level, or to the requirement for "complex sites" whatever that ends up
> being.
>
> Again, I'm not asking that we ignore certain people's problems. I'm asking
> that we think about lowering the cost. We can do that without impacting PwD
> at all, if we so choose, or we can reprioritise a few things. We'll have
> that discussion when it gets to that point. What I'm asking is that you
> take a stance on what you think about making accessibility testing with
> Silver affordable for small websites.
>
> Wilco
>
>
>
> On Wed, Sep 5, 2018 at 5:06 PM John Foliot <john.foliot@deque.com> wrote:
>
>> Supporting legislation and policy is not the purpose of accessibility
>> guidelines. People are.
>>
>>
>> Amen Charles, except...
>>
>> No matter where we collectively "draw the line", it's going to fall
>> somewhere at less than 100% accessible - that's just a reality we need to
>> accept: accessibility is a long-tail problem that ends at the individual,
>> and no standard or guideline will include every user on earth. That's
>> simply reality colliding with pragmatism.
>>
>> No matter the model, once legislations grab hold of the model there will
>> be a bright and shiny line between "pass" and "fail", even though
>> collectively we know that accessibility is far more nuanced than that.
>>
>> My concern is that a "simple" accessibility test (i.e., low cost/no cost)
>> will be a bit of a paper dragon: checking of tick boxes in pursuit of some
>> kind of truncated solution just feels wrong to me, as entities that choose
>> that route aren't really concerned about PwD, they are instead concerned
>> about not being sued. *Reaching for the lowest common denominator just
>> feels wrong, as it positions web accessibility not for its benefits, but
>> rather its liabilities.*
>>
>> I fully support free resources that help developers and content creators
>> "do better", and automated tests and other heuristic checks have their
>> place, but to create a "benchmark" based simply on free automated tests is
>> going to fail PwD as often as it benefits, with the added 'burden' of
>> falsely suggesting that because the "ACME Widget Web Site" passed all the
>> automated tests, it's "blessed" as somehow being accessible.
>>
>> The problem isn't testing, its education.
>>
>> I've long held that one of the problems with WCAG today is that most
>> people focus almost exclusively on the actual SC, and not on the Principles
>> and Guidelines. A renewed focus on the understanding piece, as opposed to
>> measuring compliance to specific checkpoints, is to me the more critical
>> piece. Silver should be focused on that, and it appears to be in many ways
>> (the desire/need for more "human readable" content is, I argue, an example
>> of the direct backlash we see/feel on focusing on the SC's and not the
>> reason behind the SC's, which results in folks struggling to decipher the
>> text of the SC's and the perception that this is "hard").
>>
>> Can individual checkpoints be created in such a way that auto or
>> programmatic testing can speed progress? I would hope so, and would support
>> that goal.
>>
>> But to establish a metric that suggested that a certain level of
>> compliance could be "claimed" based solely on auto-testing would be
>> something I would strenuously protest as being far too little, and feels
>> very retrograde to me: we moved off of WCAG 1.0 for solid reasons
>> (including the very fact that you could meet each of the WCAG 1.0
>> "tick-boxes" and still have a very inaccessible web page/site). Returning
>> to a model that repeats some of the problems from the past is going
>> backward IMHO, and I continue to fail to see how it benefits anyone except
>> entities and sites that want to appear to be doing something, yet aren't
>> prepared to do the hard miles required. It's like saying "I want to see all
>> the iconic stops along US Route 66
>> <https://www.theroute-66.com/state.html>, but I don't want to spend all
>> that time driving..." It's completely missing the bigger point.
>>
>> JF
>>
>> On Wed, Sep 5, 2018 at 8:27 AM, Hall, Charles (DET-MRM) <
>> Charles.Hall@mrm-mccann.com> wrote:
>>
>>> I wonder if we can restart this conversation with a mission or goal in
>>> mind.
>>>
>>>
>>>
>>> Instead of discussing cost of procurement, cost of testing, cost of
>>> remediation or any other associated cost generally associated with a
>>> business, can we acknowledge the cost to people without access?
>>>
>>> Accessibility guidelines are a method to aid in ensuring people can
>>> access the web.
>>>
>>>
>>>
>>> The entire and only two reasons that any public policy or legislation
>>> (anywhere in the world) has ever specifically cited the WCAG x.x is because
>>> it was the most comprehensive guidance currently available at the time the
>>> legislation was considered AND that guidance was published by the W3C as a
>>> recommendation which is considered a web standard.
>>>
>>>
>>>
>>> The only reason that any said legislation cites a compliance or
>>> conformance requirement is because a strict and explicit conformance is
>>> part of WCAG.
>>>
>>>
>>>
>>> Any new guidance that is also comprehensive and a recommendation
>>> (standard) would supersede the previous guidance when drafting any new
>>> policy or legislation. The only constant is change.
>>>
>>>
>>>
>>> Supporting legislation and policy is not the purpose of accessibility
>>> guidelines. People are.
>>>
>>>
>>>
>>> If starting from scratch, but using everything we have collectively
>>> learned from research and experience:
>>>
>>>
>>>
>>> Can we create new guidelines that support (more) people and solve all
>>> (or most of) the problems that research revealed within the current
>>> guidelines?
>>>
>>>
>>>
>>> Can we create a conformance model that is not strict and explicit to
>>> technical details, but is to human factors?
>>>
>>>
>>>
>>> Can we do that while also considering the impact on (all) stakeholders?
>>>
>>>
>>>
>>>
>>>
>>> *Charles Hall* // UX Architect, Technology
>>>
>>>
>>>
>>> charles.hall@mrm-mccann.com
>>> <charles.hall@mrm-mccann.com?subject=Note%20From%20Signature>
>>>
>>> w 248.203.8723
>>>
>>> m 248.225.8179
>>>
>>> 360 W Maple Ave,
>>> <https://maps.google.com/?q=360+W+Maple+Ave,+Birmingham+MI+48009&entry=gmail&source=g>
>>>
>>> <https://maps.google.com/?q=360+W+Maple+Ave,+%C2%A0+Birmingham+MI+48009&entry=gmail&source=g>Birmingham
>>> MI 48009
>>> <https://maps.google.com/?q=360+W+Maple+Ave,+Birmingham+MI+48009&entry=gmail&source=g>
>>>
>>>
>>> mrm-mccann.com <https://www.mrm-mccann.com/>
>>>
>>>
>>>
>>> [image: MRM//McCann]
>>>
>>> Relationship Is Our Middle Name
>>>
>>>
>>>
>>> Ad Age B-to-B Agency of the Year, 2018
>>>
>>> Ad Age Agency A-List 2016, 2017
>>>
>>> Ad Age Creativity Innovators 2016, 2017
>>>
>>> North American Agency of the Year, Cannes 2016
>>>
>>> Leader in Gartner Magic Quadrant, 2017, 2018
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From: *Mark Tanner <mktanner@btinternet.com>
>>> *Date: *Wednesday, September 5, 2018 at 7:49 AM
>>> *To: *Silver Task Force <public-silver@w3.org>
>>> *Subject: *Re: [EXTERNAL] Costs of testing with Silver
>>> *Resent-From: *Silver Task Force <public-silver@w3.org>
>>> *Resent-Date: *Wednesday, September 5, 2018 at 7:48 AM
>>>
>>>
>>>
>>> I am writing to support and slightly expand upon the points made by
>>> John. WCAG 2.0 is currently referenced throughout the EU as a key component
>>> of digital accessibility legislation.
>>>
>>> As John pointed out, the EU standard EN 301 549 lays out "Accessibility
>>> requirements suitable for public procurement of ICT products and services
>>> in Europe" http://mandate376.standards.eu/standard
>>>
>>> This standard not only references WCAG 2.0 as normative, but
>>> specifically  details all the WCAG 2.0 success criteria at reasonable
>>> length http://mandate376.standards.eu/standard/technical-requirements/#9
>>>
>>> However, as from 23 September, 2018, the standard has a wider
>>> application beyond procurement.  EU DIRECTIVE  2016/2102 on digital
>>> accessibility https://eur-lex.europa.eu/eli/dir/2016/2102/oj , which
>>> comes into legal effect in all EU countries (including the UK) requires
>>> that:
>>>
>>> ·       Public sector websites created *after *23 September 2018 need
>>> to comply with the accessibility requirements from 23 September 2019
>>>
>>> ·       Public sector websites created *before* 23 September 2018 need
>>> to comply with the accessibility requirements from 23 September 2020
>>>
>>> ·       Public sector mobile applications  need to comply with the
>>> accessibility  requirements from 23 June 2021.
>>>
>>> Paragraphs 36 and 37 of EU DIRECTIVE  2016/2102 are clear:
>>>
>>> “36. The accessibility requirements set out in this Directive … describe
>>> what must be achieved in order for the user to be able to perceive,
>>> operate, interpret and understand a website, a mobile application and
>>> related content …
>>>
>>> “37 … Those principles of accessibility are translated into testable
>>> success criteria, such as those forming the basis of the European standard
>>> EN 301 549 V1.1.2  ….Pending publication of the references to harmonised
>>> standards, or of parts thereof, in the Official Journal of the European
>>> Union, the relevant clauses of European standard EN 301 549 V1.1.2
>>> (2015-04) should be considered as the minimum means of putting those
>>> principles into practice.”
>>>
>>> As EN 301 549 currently uses WCAG 2.0 as its normative reference for all
>>> public sector websites this means that all public sector websites and
>>> applications throughout all EU countries will need to meet WCAG 2.0.
>>>
>>> It is also important to note that EN 301 549 is currently being updated
>>> to map to WCAG 2.1 to level AA). See *Explanatory Memorandum To The
>>> Public Sector Bodies (Websites And Mobile Applications) Accessibility
>>> Regulations 2018 No. 852 *http://www.legislation.gov.uk/
>>> uksi/2018/852/pdfs/uksiem_20180852_en.pdf
>>>
>>> All this clearly has important implications for any shift in the future
>>> from WCAG to Silver.
>>>
>>> Mark Tanner
>>>
>>> This message contains information which may be confidential and
>>> privileged. Unless you are the intended recipient (or authorized to receive
>>> this message for the intended recipient), you may not use, copy,
>>> disseminate or disclose to anyone the message or any information contained
>>> in the message.  If you have received the message in error, please advise
>>> the sender by reply e-mail, and delete the message.  Thank you very much.
>>>
>>
>>
>>
>> --
>> *John Foliot* | Principal Accessibility Strategist
>>
>> Deque Systems - Accessibility for Good
>>
>> deque.com
>>
>>
>
> --
> *Wilco Fiers*
> Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
>



-- 
*John Foliot* | Principal Accessibility Strategist

Deque Systems - Accessibility for Good

deque.com