Re: Costs of testing with Silver

I have a pretty strong opinion on this topic.

First, the cost of testing and cost of remediation represent different tasks – each with a large degree of variability. They both imply that web content and documents on a domain already exist. This omits all the cases where it doesn’t.

Accessibility guidelines are intended to be a set of instructions on how to create content and documents correctly in the first place, so that the variability of testing is easily controlled and remediation should be minimal to zero.

It is the role of advocacy groups and the cohort of accessibility professionals to continue to champion this message of “follow the guidelines, and do it right in the first place”. The guidelines should make it easy to do so.

Organizations should not have to run automated or manual tests to check if a caption exists or if headings are in the right order. These should be bare minimum expectations placed upon creators. At the very least, tests like that should be (and sometimes are) the standard linting of a unit test long before content and documents are public. This cost is not something organizations calculate (that I am aware of). It is simply a workflow.

If someone incurs an additional cost for testing and/or remediation, it will not be the fault of the guidelines. It will be the fault of not following web standards and accessibility guidelines. If we make the guidelines easier, it is reasonable to assume that testing is subsequently easier. If validation methods are (still) included in the guidelines, then half of each test case is already written.

I can see there being an added cost incurred by organizations that provide accessibility testing and remediation services, to update language and documentation and reports and refactor a handful of code and process. But I also see this is a fair and reasonable cost of doing business in a technology market.

The scope of Silver should include guidelines that are able to test with a method to test and a validation of that test. They may even include a score or degree or percentage of that validation. I don’t believe the scope should include considerations for cost. It has always been discussed that Silver should consider how it is used by various stakeholders. But “how” intentionally includes a little ambiguity. If I hire 10 people to train 1000 people on Silver for 30 minutes, that has everything to do with a business decision and nothing to do with the actual guidelines. If I write 11 new test cases that take 3 minutes each per page of a site, I find a way to make up that time elsewhere and improve upstream decisions that allow me run them less and find zero defects, and I get a better product.

The economic impact of accessibility is an argument we all use in advocacy. That should be enough to continue to win cost arguments.

Cheers,


Charles Hall // UX Architect, Technology

charles.hall@mrm-mccann.com<mailto:charles.hall@mrm-mccann.com?subject=Note%20From%20Signature>
w 248.203.8723
m 248.225.8179
360 W Maple Ave, Birmingham MI 48009
mrm-mccann.com<https://www.mrm-mccann.com/>

[MRM//McCann]
Relationship Is Our Middle Name

Ad Age B-to-B Agency of the Year, 2018
Ad Age Agency A-List 2016, 2017
Ad Age Creativity Innovators 2016, 2017
North American Agency of the Year, Cannes 2016
Leader in Gartner Magic Quadrant, 2017, 2018



From: Luis Garcia <w3c@garcialo.com>
Date: Tuesday, August 28, 2018 at 2:30 PM
To: Jeanne Spellman <jspellman@spellmanconsulting.com>
Cc: Silver Task Force <public-silver@w3.org>
Subject: Re: [EXTERNAL] Costs of testing with Silver
Resent-From: Silver Task Force <public-silver@w3.org>
Resent-Date: Tuesday, August 28, 2018 at 2:29 PM

I'd love to make "reduce the cost of accessibility testing" a goal. That said, is there already a way that "cost of accessibility testing" has been calculated for WCAG 1.0, 2.0, and 2.1?

If it's not something that already exists, I don't know that we'd want to have it be a primary goal. As it is, there will already be a cost associated with adjusting to Silver.

I think we might be able to lower the overall cost of accessibility by making it more a part of everyone's job. And I think we can do that by making the guidelines more accessible.

luis

On Tue, Aug 28, 2018 at 7:55 AM Jeanne Spellman <jspellman@spellmanconsulting.com<mailto:jspellman@spellmanconsulting.com>> wrote:

This is a very interesting idea that I do think we need to discuss.  When I was first thinking about Silver two years ago, I had a thought that we could do automated testing for a basic level of accessibility.  As we went through all the research and started forming the ideas and proposals, I had forgotten about it.  I am open to looking at this in more detail.

My concern would be the amount of disability needs that could be included in reduced-cost testing, particularly the non-physical needs, like cognitive disabilities.  I know there is new research and testing in the last couple years that could be a viable solution.

I would like to schedule this discussion so we could have a number of people involved who care about this issue, and devote an entire meeting (or more) to it.

On 8/28/2018 6:19 AM, Wilco Fiers wrote:
Hey all,

Firstly, all hats off. Sharing a personal view here. I wanted to reach out about a thing that I've been concerned with regarding Silver. With WCAG 2.1 I saw some discussions about the increased cost of testing compared to WCAG 2.0. Thinking about the adoption of WCAG 2.0 from WCAG 1.0, there too I saw that the amount of work it took to do accessibility testing had increased quite significantly.

I'm concerned that (as far as I can tell) there hasn't yet been a discussion about costs of testing with Silver. I know its still early days, but I think that we should have that discussion, and decide what kind of target we'd like to hit for Silver. There are all these fantastic ideas floating around, about score cards, usability testing, expanding to include non-web technologies. Lots of great stuff, but we have to be aware that all of these things are going to have a price tag.

I would very much like for the Silver group to decide how much they think the cost of doing accessibility testing is allowed to increase. Is it okay for the costs of testing to double between WCAG 2.0 and Silver like they did from WCAG 1.0 to 2.0? Is it allowed to increase at all? Should Silver be designed to decrease costs instead?

Making Silver easier to use, lowering the barrier to entry, those are fantastic goals. But those things really don't matter if someone can't get the budget to do accessibility testing. And without testing, you can't maintain an accessible site. I don't think it's unreasonable to think that if Silver decreases the cost of accessibility testing, it could get wider adoption than WCAG 2 did. Where is if the opposite happens, if testing for Silver is far more expansive than it is for WCAG 2, that organisations might just stick with WCAG 2 for a long time to come.

Personally, I'm of the opinion that WCAG 2 is already too expansive. When I was still testing WCAG 1, I'd regularly test websites of smaller organisations. Those organisations stopped coming when the costs went up for WCAG 2. I think a good target for Silver would be that at the bronze level, costs for testing are about half what they are for WCAG 2.0 Level AA. I believe that that would make it affordable again for small businesses, which I think should be a goal for Silver.

Either way, Silver needs to be designed with an eye on testing costs, and it would help if we had some goals defined for it.

--
Wilco Fiers
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
Error! Filename not specified.

This message contains information which may be confidential and privileged. Unless you are the intended recipient (or authorized to receive this message for the intended recipient), you may not use, copy, disseminate or disclose to anyone the message or any information contained in the message.  If you have received the message in error, please advise the sender by reply e-mail, and delete the message.  Thank you very much.