Re: Costs of testing with Silver

Hi Wilco,

> The question is, how do we enable organisations with a small budget to
still use Silver?

Actually, while that is indeed a question worth asking, I question whether
budgets should be a primary determiner for how we create our
Recommendation.

Our mandate is not to save money, it's to ensure we've provided measurable
standards that we can categorically claim will increase the accessibility
of a page - if not for *all* users, for a larger percentage of users. We
need to be mindful of costs, yes, but if we start crafting the guidelines
based on cost, we're (I argue) looking in the wrong direction.

Here in America, you can go to some restaurants and buy a "hamburger" for
$0.99. Now, if you are prepared to accept that your hamburger only cost you
$0.99 and you still consider that a "good meal", well... cost is
subjective, and you get what you pay for. If all you are prepared to spend
is a buck, then you will get a buck's worth of "hamburger", and nothing
more. I think additionally that part of the problem is that you aren't
actually talking about "testing", but rather "conformance evaluation",
which, while very similar, is not the same.

Here's an analogy: I am walking down the street and see my Doctor. He asks,
"How are you feeling?" I reply, "Pretty good, thanks." The doctor responds,
"Well, you're looking good. I notice you've gained a few pounds - you
should be careful there". Me, "Yep, I hear you. OK doc, see you around"
Doctor, "OK, see you."

Now, all of that was "free", but it wasn't a medical examination. For a
medical examination, I need to go to the doctor's office and have a proper
check-up. There is no getting around that simple fact. However, each exam
does not require an MRI scan, a full blood screen test, having me run on a
treadmill, etc., and if my doctor's visits are frequent and regular, the
doctor's exam is a check-up, monitoring and looking for changes. So the
best way for me to contain medical costs is to see my doctor frequently.
Yes, there is a cost associated to that, but the cost is minimized because
I see the doctor routinely.

You and I both know that the longer you wait to test a project, the more
testing will be involved, and likely the number of issues uncovered will
grow exponentially the longer you wait to test. If the only time you are
doing accessibility testing is 2 days before product launch - be it 5 pages
or 5,000 pages, you waited too long - plain and simple.

So perhaps some of the things we might or should be thinking about is a) an
educational piece that teaches to test early, test often, b) a reminder
that accessibility testing isn't like QA testing - it needs to be
integrated into the design and development process and not done at the end
of the process, c) accessibility testing isn't an *OBLIGATION*, but rather
a *Feature*, a benefit that makes any site not only "accessible" but more
user-friendly for all users, and thus of greater value.

Early on, long before my days at Stanford, or JPMC, or Deque, I ran a
small, one-man web-dev shop operating in Ottawa. BAck then, I applied all
of the known principles of accessibility to each site I created. I did the
testing, not as a cost line-item on the final invoice, but rather as part
of what it was I offered and sold. I did it as a matter of
professional pride, and I up-sold the fact that my sites performed well in
search engine rankings (which I could prove), and worked for a larger group
of users. I didn't make "accessibility" something special or unique, and it
certainly wasn't an option when contracting - it was simply part of the
package and my dev practice. If there are shops today that sell
"accessibility" as an additional option, then the problem isn't with our
recommendation or testing strategies, it's an institutional problem at the
smaller shop that can only be corrected through education.

Wilco, at the end of the day, we'll never stop restaurants from selling
$0.99 hamburgers, because there will always be somebody looking for
"cheaper". If a smaller shop wants to cut corners in their dev and testing
processes, we are not in any position to impact that. In some territories,
the enforcement is left to regulatory bodies outside of the W3C. In other
territories, there is no legal mandate, but again, that isn't something
that the W3C can change. We provide the "recipe" for how to make the best,
most delicious and "accessible" hamburger, but in the end, we neither make
the hamburger, nor decide on which part of the recipe others will or won't
ignore, and shops will price their 'burger at whatever price they want.
That may not be ideal, but it's real.

JF

On Thu, Sep 6, 2018 at 7:09 AM, Wilco Fiers <wilco.fiers@deque.com> wrote:

> David, Mark,
> Not to ignore your suggestion of putting together a document, but I think
> we need to have at least some common idea of what to do about this cost
> question before we can put anything together. I'm happy to work on
> something once we've gotten closer to a solution, but I think this
> discussion needs to run its course first.
>
> @John, you've raised a lot of concerns now, but I haven't really heard any
> alternative suggestions from you yet. The question is, how do we enable
> organisations with a small budget to still use Silver?
>
> Another perspective I want to add is that organisations are already using
> "light" versions of WCAG. Even the W3C has gotten in on that game with the
> easy checks (https://www.w3.org/WAI/test-evaluate/preliminary/). Many
> organisations use approaches like this. The question isn't if these types
> of tests should exist, but if we as a community want to recommend how to do
> this in a way that doesn't leave big gaps.
>
> Here's the thing. If we don't create a conformance level in Silver that is
> affordable to small businesses, either they'll do nothing instead, or
> they'll use some third party "light" testing method that didn't go through
> the quality controls of the W3C consensus process.
>
> It seems to me like Silver should be achievable not just for the top 5% of
> websites. If you want to talk percentages, this is the discussion we should
> have. How many websites do we think should conform to Silver? Knowing if a
> site conforms required at least one test in the lifetime of a website. Ive
> created a Google spreadsheet to help us see how the cost of audits
> disproportionally impact low budget websites:
>
> https://docs.google.com/spreadsheets/d/144ddhHfBUbRtWEoPdJ8MqsBDq0wzH
> DHgWJ-yL-fH7Ow/edit?usp=sharing (Let me know if anyone wants me to share
> a plain HTML version of these tables)
>
> Feel free to play around with the numbers. I think this shows quite well
> how 1) small sites need to reserve a disproportionately high budget for
> accessibility testing, and 2) how this could be solved by making
> adjustments for complexity of the website in Silver.
>
> Wilco
>
>


-- 
*John Foliot* | Principal Accessibility Strategist

Deque Systems - Accessibility for Good

deque.com