Re: Costs of testing with Silver

Hey all,
Moving this discussion back a bit again. The basis for my question still
doesn't seem to have come across very well. So here it is put in different
terms.

Most countries do not have legal requirements to develop accessible
websites. In many cases web accessibility is effectively voluntary. This
has been true in the Netherlands (where I'm based) so I'll speak from that.
But most of Europe looks like this too. Anne and Alastair, also based in
Europe, have confirmed as much in their e-mails.

The organisations I'm especially concerned about are smaller organisations
that have their websites developed externally. They have three options when
it comes to accessibility.
1) They trust the web developer to make the site accessible.
2) They have someone internally learn enough about accessibility to test
the site
3) They hire an external expert to do a one-time test of the website.

In reverse order. 3) When I started testing WCAG 2.0 instead of WCAG 1.0,
prices went up. Significant enough that small organisations stopped having
their sites tested. These are organisations in the 10 - 50 employees range.
WCAG 2.0 testing is costly enough that those organisations couldn't afford
it anymore. That left them #2 and #1. Seeing is they couldn't build their
own website, learning enough about accessibility that they can test isn't
really an option. We can "plain language" Silver all we want, in the end it
does involve starting up a few screen readers and being familiar enough
with them that you can do basic testing.

That only leaves option #1, ask developers to make sites accessible and
hope they can do it. That's not a thing that worked either. In research
that the Dutch government did they looked at all self declarations of
accessibility that Dutch municipalities did. Out of all of them, only a
single one turned out to be valid.

The costs of testing WCAG 2.0 has lead to disability advocacy organisations
to stop having their sites tested for accessibility. It sounds like Anne
has seen the same happen in Denmark. We can't pretend that costs don't
matter. Web accessibility is voluntary for the majority of the world. If we
can't come up with an entry level that is affordable even for small
businesses, those businesses just aren't ever going to look at
accessibility.

I'm not advocating that we ignore the needs of some people. None of this
means we have to ignore the needs of people with disabilities. What it
might mean is that we ignore the needs of some developers to have their
edge cases. We may also add some requirements that simplify how things are
tested. For instance, don't skip heading levels, and make sure links to the
same page have the same link text. Or that within a form / fieldset, all
form fields have a unique name. There are things we can do at the start
level so that we can move some of the more resource-intensive testing to a
higher level.

There is an absolutely massive discrepancy between how much time it takes
to see if a site is mostly accessible, and testing a site using WCAG. Most
experts can look at a site for 5 minutes and tell you if that site has
structural accessibility issues. It takes another 2 days of testing to
decide if that site conforms to WCAG-EM. Two days is enough time to build a
simple website. We really should be able to come up with a testing approach
that doesn't require more time to test a site than it does to build it.

Wilco


On Tue, Sep 4, 2018 at 10:37 PM Lisa Snider <
lisa@accesschangeseverything.com> wrote:

> Hi John,
>
>
>
> I can’t say more, but I know of three pieces of legislation that will
> likely not include WCAG, or any standards, in them. I can’t tell you which
> ones, as they haven’t gone through the last stages of approval to make them
> official (I can’t even say the country as they would make me disappear!).
> They may morph at the last stages, or not, that is why I can’t say any more
> than I have in this email. Plus, as we all know governments change and that
> could factor into things as well.
>
>
>
> I just wanted to throw that into the mix, because what happened years ago
> isn’t necessarily what will happen today. So, all I am saying is to keep
> this mind in terms of cost factors.
>
>
>
> Cheers
>
>
>
> Lisa
>
>
>
>
>
>
>
> Lisa Snider
>
> Senior Digital Accessibility Consultant & Trainer
>
> Access Changes Everything
>
> Web: www.accesschangeseverything.com
>
> Phone: (800) 208-1936
>
>
>
> *From: *John Foliot <john.foliot@deque.com>
> *Date: *Tuesday, September 4, 2018 at 3:30 PM
> *To: *Lisa Snider <lisa@accesschangeseverything.com>
> *Cc: *Alastair Campbell <acampbell@nomensa.com>, "public-silver@w3.org" <
> public-silver@w3.org>
> *Subject: *Re: Costs of testing with Silver
>
>
>
> Hi Lisa,
>
>
>
> I'm not sure where you are getting your research from, but I'm gong to
> question that statement.
>
>
>
> Perhaps Canadian provincial legislation might not reference WCAG directly
> (such as AODA), but one of the key reasons for the way WCAG 2.1 was
> structured and developed was to ensure it did NOT impact the Section 508
> refresh (which maps directly back to WCAG), a point of significant concern
> to more than one legal entity that commented early on the process.
>
> E205 Electronic Content
>
> E205.1 General. Electronic content shall comply with E205.
>
> E205.2 Public Facing. Electronic content that is public facing shall
> conform to the accessibility requirements specified in E205.4.
>
>
>
> E205.4 Accessibility Standard. Electronic content shall conform to Level A
> and Level AA Success Criteria and Conformance Requirements in WCAG 2.0
> (incorporated by reference, see 702.10.1).
>
> (source:
> https://www.access-board.gov/guidelines-and-standards/communications-and-it/about-the-ict-refresh/final-rule/text-of-the-standards-and-guidelines#E205-content
> )
>
>
>
> Additionally, caution was also exercised to ensure our work did not
> negatively impact the European Union's *EN 301 549 - Accessibility
> requirements suitable for public procurement of ICT products and services
> in Europe, - V1.1.2* which *normatively references WCAG*:
>
>
>
> *Normative references*
>
>
>
> The following referenced documents are necessary for the application of
> the present document.
>
>
>    1. ETSI ETS 300 381: "Telephony for hearing impaired people; Inductive
>    coupling of telephone earphones to hearing aids".
>    2. ETSI ES 200 381-1: "Telephony for hearing impaired people;
>    Inductive coupling of telephone earphones to hearing aids Part 1:
>    Fixed-line speech terminals".
>    3. ETSI ES 200 381-2: "Telephony for hearing impaired people;
>    Inductive coupling of telephone earphones to hearing aids; Part 2: Cellular
>    speech terminals".
>    4. *W3C Recommendation (11 December 2008)/ISO/IEC 40500:2012: "Web
>    Content Accessibility Guidelines (WCAG) 2.0 ".*
>
> (source: http://mandate376.standards.eu/standard/references)
>
>
>
> This seems to contradict your assertion to some extent, for while these
> recent legislations do not in fact re-publish WCAG 2.0, they both
> normatively reference that Recommendation as *the* standard upon which
> success or failure will be judged against (at least for web/digital
> content).
>
>
>
> A lot of time and effort has been applied towards international
> harmonization of accessibility standards and laws, and I for one would be
> quite surprised if that activity would be jettisoned in favor of a Tower of
> Babel approach to legislated web accessibility.
>
>
>
> JF
>
>
>
> On Tue, Sep 4, 2018 at 2:53 PM, Lisa Snider <
> lisa@accesschangeseverything.com> wrote:
>
> I will say again that laws are now not necessarily going to include
> specific standards in them, the trend is to take them out, and have them in
> a separate place, not in the law themselves. I can’t say any more than that
> until things are official in three laws I know about…
>
>
>
> This means that people may not use WCAG, or any other standards, or use a
> blending on them. So the cost theory may not even apply anymore, in terms
> of this discussion, because it becomes cost in general and not linked to
> any one standard
>
>
>
> Please keep that in mind when you go forward.
>
>
>
> Cheers
>
>
>
> Lisa
>
>
>
> Lisa Snider
>
> Senior Digital Accessibility Consultant & Trainer
>
> Access Changes Everything
>
> Web: www.accesschangeseverything.com
>
> Phone: (800) 208-1936
>
>
>
> *From: *Alastair Campbell <acampbell@nomensa.com>
> *Date: *Tuesday, September 4, 2018 at 8:57 AM
> *To: *"public-silver@w3.org" <public-silver@w3.org>
> *Subject: *Re: Costs of testing with Silver
> *Resent-From: *<public-silver@w3.org>
> *Resent-Date: *Tue, 04 Sep 2018 13:57:08 +0000
>
>
>
> Charles wrote
>
> > Conformance to accessibility guidelines is not a law. Accessibility is.
>
>
>
> That varies, several countries / regions use WCAG as part of their laws.
> E.g. Section 508 in the US, and Mandate 376 in the EU are both procurement
> laws that incorporate WCAG 2.
>
>
>
> The general law in the UK better fits your statement, although the
> guidelines become a reference point in cases.
>
>
>
> The upshot of that is that the guidelines should be legal-compatible. I.e.
> not driven by legal factors, but possible to use in policy / governance /
> legal cases. I don’t think they would have been as successful without that
> aspect.
>
>
>
>
>
> > Conformance criteria of WCAG 2.x are difficult according to all of the
> research conducted.
>
>
>
> I’m not sure if you mean implementing, testing or understanding here?
>
> There are things which are difficult such as covering certain
> user-requirements, and in understanding by non-technical audiences.
> However, assuming that people understand them I don’t think the research
> was saying they were difficult to implement or test, unless I missed
> something?
>
>
>
>
>
> > Silver is prototyping different methods of conformance that are less
> difficult.
>
>
>
> If “less difficult” means easier to understand and able to provide better
> coverage, yes. However, the criteria themselves may not be easier to
> implement or test.
>
>
>
>
>
> > Cost is not as important as results that support and improve those
> purposes.
>
>
>
> Indeed, but I would consider a ‘total cost of ownership’ model, i.e. what
> is the overall impact to all people with disabilities.
>
>
>
> To take extreme ends of the scale:
>
>    1. If all the criteria were easy to implement and test with minimal
>    overhead, we’d have fantastic take-up but terrible coverage. (Similar to
>    what automated testing provides now.)
>
>    2. If some of the criteria took a significant (>30%) proportion of
>    project budgets to implement/test, we could have good coverage and terrible
>    take-up. If it’s enough to prevent policy makers from choosing it as the
>    standard for public & private organisations, you’d have very little take-up.
>
>
>
> So I’d be very cautious about saying that any criteria would get in
> regardless of feasibility/cost.
>
>
>
> Where Silver has an opportunity to improve the situation is:
>
>    - Providing means of passing criteria that are not a simple yes/no
>    answers.
>    - Providing levels that can be applied differently by different
>    organisations.
>
>
>
> For example, some criteria might only be applicable to larger
> organisations (to be defined), or those aiming for a higher level of
> accessibility / inlusiveness. We can take the current WCAG 2.1 AA as a
> baseline and increase coverage in feasible ways with a new approach.
>
>
>
> Cheers,
>
>
>
> -Alastair
>
>
>
>
>
> --
>
> *John Foliot* | Principal Accessibility Strategist
>
> Deque Systems - Accessibility for Good
>
> deque.com
>
>
>


-- 
*Wilco Fiers*
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG