Re: Costs of testing with Silver

Hey Charles,
Let me verify that I understand your position. So you are suggesting that
costs of accessibility testing should not factor into the considerations
for Silver at all? So in a hypothetical situation where accessibility
testing from Silver ends up costing 5 times as much as an existing WCAG 2.0
Level AA audit, that would still be acceptable for you? What about if it
ends up as 20 times the cost of a WCAG 2.0 Level AA audit? I would wager a
guess that there is an upper limit to what you think is acceptable costs
for accessibility audits. For most people there would be, so lets talk
about it and see what we think it is.

I see your point of down stream benefits, but I don't think that is going
to change from WCAG 2.0 to Silver. If testing with Silver costs 5 times
what it does to test WCAG 2.0, but the down stream benefits are the same,
Silver is going to be a difficult sell. I want to distinguish here between
two arguments. My first one is that testing costs matter, and we should set
ourselves some goals. To Luis point, I don't think there are current
estimates, but we can most certainly come up with them if we ask different
organisations to give us some numbers. Man-hours per page for example?

My second argument, that we should seriously consider lowering the cost of
testing Silver at Bronze level, from testing WCAG 2.0 Level AA, is this.
For most organisations, especially for smaller organisations, an
accessibility audit is the first thing they do once they've decided they
want to invest in making their digital content more accessible. It is their
first opportunity to learn about accessibility, to develop some basic
knowledge about how web accessibility applies to their digital assets.
We've all seen cases where people thought they were doing just fine until
they saw that first audit report come back. Accessibility audits are a key
tool in our arsenal for raising awareness and developing basic
accessibility skills. For most smaller organisations it is often the only
thing they'll ever invest in when it comes to accessibility.

If Silver is going to make accessibility testing more expansive, than it is
not unreasonable to expect that fewer organisations will make that initial
investment in having an audit done. I think that lowering the costs of
accessibility audits lowers the barrier to entry for Silver. That is
already a goal the group set itself. When I moved from testing WCAG 1.0 to
WCAG 2.0, I saw a lot of smaller organisations stop working on
accessibility, because they couldn't afford it anymore. Testing is an
absolutely essential part of doing accessibility. If you don't test, you
don't know how accessible something is. Audits are often the only tool
small businesses have to verify the quality of websites that were developed
by a third party. If those are not affordable, if accessibility audits cost
10% or more of the entire budget for developing a new website,
accessibility testing just doesn't happen. That kind of money doesn't make
business sense unless there's a legal risk to it. We're already at that
level with WCAG 2.0. I've seen those kinds of number.

Wilco

On Wed, Aug 29, 2018 at 2:53 AM Hall, Charles (DET-MRM) <
Charles.Hall@mrm-mccann.com> wrote:

> I have a pretty strong opinion on this topic.
>
>
>
> First, the cost of testing and cost of remediation represent different
> tasks – each with a large degree of variability. They both imply that web
> content and documents on a domain already exist. This omits all the cases
> where it doesn’t.
>
>
>
> Accessibility guidelines are intended to be a set of instructions on how
> to create content and documents correctly in the first place, so that the
> variability of testing is easily controlled and remediation should be
> minimal to zero.
>
>
>
> It is the role of advocacy groups and the cohort of accessibility
> professionals to continue to champion this message of “follow the
> guidelines, and do it right in the first place”. The guidelines should make
> it easy to do so.
>
>
>
> Organizations should not have to run automated or manual tests to check if
> a caption exists or if headings are in the right order. These should be
> bare minimum expectations placed upon creators. At the very least, tests
> like that should be (and sometimes are) the standard linting of a unit test
> long before content and documents are public. This cost is not something
> organizations calculate (that I am aware of). It is simply a workflow.
>
>
>
> If someone incurs an additional cost for testing and/or remediation, it
> will not be the fault of the guidelines. It will be the fault of not
> following web standards and accessibility guidelines. If we make the
> guidelines easier, it is reasonable to assume that testing is subsequently
> easier. If validation methods are (still) included in the guidelines, then
> half of each test case is already written.
>
>
>
> I can see there being an added cost incurred by organizations that provide
> accessibility testing and remediation services, to update language and
> documentation and reports and refactor a handful of code and process. But I
> also see this is a fair and reasonable cost of doing business in a
> technology market.
>
>
>
> The scope of Silver should include guidelines that are able to test with a
> method to test and a validation of that test. They may even include a score
> or degree or percentage of that validation. I don’t believe the scope
> should include considerations for cost. It has always been discussed that
> Silver should consider how it is used by various stakeholders. But “how”
> intentionally includes a little ambiguity. If I hire 10 people to train
> 1000 people on Silver for 30 minutes, that has everything to do with a
> business decision and nothing to do with the actual guidelines. If I write
> 11 new test cases that take 3 minutes each per page of a site, I find a way
> to make up that time elsewhere and improve upstream decisions that allow me
> run them less and find zero defects, and I get a better product.
>
>
>
> The economic impact of accessibility is an argument we all use in
> advocacy. That should be enough to continue to win cost arguments.
>
>
>
> Cheers,
>
>
>
>
>
> *Charles Hall* // UX Architect, Technology
>
>
>
> charles.hall@mrm-mccann.com
> <charles.hall@mrm-mccann.com?subject=Note%20From%20Signature>
>
> w 248.203.8723
>
> m 248.225.8179
>
> 360 W Maple Ave, Birmingham MI 48009
>
> mrm-mccann.com <https://www.mrm-mccann.com/>
>
>
>
> [image: MRM//McCann]
>
> Relationship Is Our Middle Name
>
>
>
> Ad Age B-to-B Agency of the Year, 2018
>
> Ad Age Agency A-List 2016, 2017
>
> Ad Age Creativity Innovators 2016, 2017
>
> North American Agency of the Year, Cannes 2016
>
> Leader in Gartner Magic Quadrant, 2017, 2018
>
>
>
>
>
>
>
> *From: *Luis Garcia <w3c@garcialo.com>
> *Date: *Tuesday, August 28, 2018 at 2:30 PM
> *To: *Jeanne Spellman <jspellman@spellmanconsulting.com>
> *Cc: *Silver Task Force <public-silver@w3.org>
> *Subject: *Re: [EXTERNAL] Costs of testing with Silver
> *Resent-From: *Silver Task Force <public-silver@w3.org>
> *Resent-Date: *Tuesday, August 28, 2018 at 2:29 PM
>
>
>
> I'd love to make "reduce the cost of accessibility testing" a goal. That
> said, is there already a way that "cost of accessibility testing" has been
> calculated for WCAG 1.0, 2.0, and 2.1?
>
>
>
> If it's not something that already exists, I don't know that we'd want to
> have it be a primary goal. As it is, there will already be a cost
> associated with adjusting to Silver.
>
>
>
> I think we might be able to lower the overall cost of accessibility by
> making it more a part of everyone's job. And I think we can do that by
> making the guidelines more accessible.
>
>
>
> luis
>
>
>
> On Tue, Aug 28, 2018 at 7:55 AM Jeanne Spellman <
> jspellman@spellmanconsulting.com> wrote:
>
> This is a very interesting idea that I do think we need to discuss.  When
> I was first thinking about Silver two years ago, I had a thought that we
> could do automated testing for a basic level of accessibility.  As we went
> through all the research and started forming the ideas and proposals, I had
> forgotten about it.  I am open to looking at this in more detail.
>
> My concern would be the amount of disability needs that could be included
> in reduced-cost testing, particularly the non-physical needs, like
> cognitive disabilities.  I know there is new research and testing in the
> last couple years that could be a viable solution.
>
> I would like to schedule this discussion so we could have a number of
> people involved who care about this issue, and devote an entire meeting (or
> more) to it.
>
>
>
> On 8/28/2018 6:19 AM, Wilco Fiers wrote:
>
> Hey all,
>
>
>
> Firstly, all hats off. Sharing a personal view here. I wanted to reach out
> about a thing that I've been concerned with regarding Silver. With WCAG 2.1
> I saw some discussions about the increased cost of testing compared to WCAG
> 2.0. Thinking about the adoption of WCAG 2.0 from WCAG 1.0, there too I saw
> that the amount of work it took to do accessibility testing had increased
> quite significantly.
>
>
>
> I'm concerned that (as far as I can tell) there hasn't yet been a
> discussion about costs of testing with Silver. I know its still early days,
> but I think that we should have that discussion, and decide what kind of
> target we'd like to hit for Silver. There are all these fantastic ideas
> floating around, about score cards, usability testing, expanding to include
> non-web technologies. Lots of great stuff, but we have to be aware that all
> of these things are going to have a price tag.
>
>
>
> I would very much like for the Silver group to decide how much they think
> the cost of doing accessibility testing is allowed to increase. Is it okay
> for the costs of testing to double between WCAG 2.0 and Silver like they
> did from WCAG 1.0 to 2.0? Is it allowed to increase at all? Should Silver
> be designed to decrease costs instead?
>
>
>
> Making Silver easier to use, lowering the barrier to entry, those are
> fantastic goals. But those things really don't matter if someone can't get
> the budget to do accessibility testing. And without testing, you can't
> maintain an accessible site. I don't think it's unreasonable to think that
> if Silver decreases the cost of accessibility testing, it could get wider
> adoption than WCAG 2 did. Where is if the opposite happens, if testing for
> Silver is far more expansive than it is for WCAG 2, that organisations
> might just stick with WCAG 2 for a long time to come.
>
>
>
> Personally, I'm of the opinion that WCAG 2 is already too expansive. When
> I was still testing WCAG 1, I'd regularly test websites of smaller
> organisations. Those organisations stopped coming when the costs went up
> for WCAG 2. I think a good target for Silver would be that at the bronze
> level, costs for testing are about half what they are for WCAG 2.0 Level
> AA. I believe that that would make it affordable again for small
> businesses, which I think should be a goal for Silver.
>
>
>
> Either way, Silver needs to be designed with an eye on testing costs, and
> it would help if we had some goals defined for it.
>
>
>
> --
>
> *Wilco Fiers*
>
> Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
>
> *Error! Filename not specified.*
>
>
>
> This message contains information which may be confidential and
> privileged. Unless you are the intended recipient (or authorized to receive
> this message for the intended recipient), you may not use, copy,
> disseminate or disclose to anyone the message or any information contained
> in the message.  If you have received the message in error, please advise
> the sender by reply e-mail, and delete the message.  Thank you very much.
>


-- 
*Wilco Fiers*
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG