Re: Costs of testing with Silver

Hi all,

I share the concern Wilco originally wrote out about. A lot of other great arguments went into the discussion afterwards, but for me I think the important distinction when it comes to cost of testing is between the cost of initial uptake for the new guidelines and subsequent operation of accessibility testing.

The requirements for Silver should help ensure that the learning curve for the initial uptake is as flat as possible, which should increase the cost of initial uptake.
Living in a country where accessibility testing is something you buy from either consultants or tool vendors, this cost will mostly be a one-time cost placed with specialized companies. This is not my concern.

My concern is that if conformance to Silver can only be confirmed by e.g. user testing with X number of users, or that the conformance model prescribes that you can only confirm 5% of your conformance score through automated testing (where it might be possible to automate e.g. 30% today – and even more in the future, as new technologies mature), the price of performing a conformance test in the subsequent operations phase will increase, and the bill for this will have to go to the organization procuring the test.
As Wilco noted, this will mean that having their own website tested will be out of reach for smaller companies, but also for e.g. disability organizations running on small budgets.

I don’t think the need for measuring accessibility conformance will decrease if the guidelines are easier to understand.
Speed limits are quite easy to understand, but the police still wants to measure if drivers stick to them. And even if there are no police around, drivers will be keeping an eye on the speedometer, measuring themselves to ensure they are behaving as they see fit. I believe the same needs exist for accessibility.

I hope that Silver will make not only accessibility available to a wider audience, but also accessibility testing.
And in jurisdictions where organisations can be obliged to prove their accessibility conformance, it is necessary that this obligation will not bring the organization to its knees.

In the future we are bound to add more “success criteria” to cover a broader spectrum of disabilities, and as I see it, we need to find some way of countering for this to keep the price tag for conformance testing from skyrocketing.
We need to allow for people to work smarter, not harder.

So my question for Silver is:

How might we build a model that ensures conformance claims can be made in a cost-efficient way as we keep adding new “success criteria” to the guidelines?

Best Regards,


Anne Thyme Nørregaard
Digital Accessibility Product Expert

[cid:image002.png@01D3AA4A.C3084FC0]<http://siteimprove.com/da/>

Sankt Annæ Plads 28  |  DK-1250 København K
Mobile +45 23 28 53 91  |  
ath@siteimprove.com<mailto:ath@siteimprove.com>

Facebook<https://www.facebook.com/Siteimprove>   Twitter<https://twitter.com/Siteimprove>   LinkedIn<https://www.linkedin.com/company/siteimprove>

Unsubscribe<http://go.siteimprove.com/hs/manage-preferences/unsubscribe-simple>



From: John Foliot <john.foliot@deque.com>
Date: Thursday, 30 August 2018 at 19.21
To: Alastair Campbell <acampbell@nomensa.com>
Cc: "Hall, Charles (DET-MRM)" <Charles.Hall@mrm-mccann.com>, "public-silver@w3.org" <public-silver@w3.org>
Subject: Re: Costs of testing with Silver
Resent-From: <public-silver@w3.org>
Resent-Date: Thursday, 30 August 2018 at 19.21

So this is where I was lazily using ‘cost’ as an outcome of something that is not feasible or testable.
OK, now I understand. Might I suggest that in *this* conversation, a merging of those ideas is counter-productive (or at least confusing), as Wilco started this thread specifically about the CO$T of testing, and not of feasibility or testability.

For a specific example, you might recall the discussion about non-text contrast, where the difficulty (and presumably cost) was cited as a reason not to include that SC.
Sorry Alastair, I don't recall that. Further, we *do* have a new SC regarding non-text contrast, so I am unclear what you are suggesting here. I know you and I had a disagreement (ongoing I might add) about whether or not native focus indication on a modified background color should or would be in scope, and I believe you may have cited cost of remediation as a reason for accepting dotted back lines on a black background in Firefox as "acceptable" or excluded from that SC (even though I still disagree...)

There were multiple COGA SC where the difficulty (and therefore cost) of testing were factors, such as plain language. IIRC, we both raised that.
Actually, my concern then was around testability and scalability, which we could not get agreement on. Many of the COGA folk were concerned that "reading level" was not the right test, yet as I recall, no other testable method was brought forward. I do not however remember "cost" being part of that discussion.

Again, while we need to be mindful of cost, that cannot nor should not be a barrier to advancement in and of itself. As I previously stated, everything has a cost, and we (at least *I*) are not here for legislation or corporate policy reasons - we are here to create the right technical requirements and guidance to benefit people with disabilities.

Retro-fitting a building built in the 18th century to include providing a lift for people in wheel chairs has a cost. Should physical build requirements then not include 18th century buildings as requiring accommodation to those people? And if the job is to create architectural standards to ensure those lifts are built, we again need to be mindful of cost, but not necessarily governed or directed by cost, is all I am saying.

JF


On Thu, Aug 30, 2018 at 12:01 PM, Alastair Campbell <acampbell@nomensa.com<mailto:acampbell@nomensa.com>> wrote:
Hi John,

JF wrote:
> Like others, I've been lurking on this conversation. At this time, I have to side primarily with Charles Hall and David Sloan.

I’m not particularly disagreeing in terms of goals, just trying to pin down the how.

We might have to tease out differences between cost, feasibility & testability. For me ‘cost’ is an umbrella for those. Something that is less feasible is more costly etc.

> While the Bronze, Silver and Gold levels has some merit and worthy of consideration, it has not been adopted as "the way forward" has it?
Not specifically, but given the requirements for more flexibility and measurable & task-based assessment, that’s a mechanism to use during the prototyping.

> I am unsure where Alastair is coming up with this assertion. As experienced professionals in this field, again, we understand that there are costs involved, but cost was never a formal consideration during any of our work (that I recall).

So this is where I was lazily using ‘cost’ as an outcome of something that is not feasible or testable. For a specific example, you might recall the discussion about non-text contrast, where the difficulty (and presumably cost) was cited as a reason not to include that SC.

There were multiple COGA SC where the difficulty (and therefore cost) of testing were factors, such as plain language. IIRC, we both raised that.

Accessible authentication didn’t get in due to concerns about feasibility. There already were expensive options for supporting it, but that wasn’t enough, there had to be reasonable solutions that work across any website. Hence that was a “cost” consideration.


> one of my other concerns:  that certain disabilities will remain "left out", not due to our ability to address their needs, but to do so has a "cost" that some organizations will consider onerous.

Did you see where I suggested we do include all the requirements? That’s a structural change from WCAG 2.x in order to make sure no disabilities are left out.

There will be gaps at the technique/criteria level where there are not feasible ways for every website to meet the user-requirements, at least initially. That structure would highlights the gaps, and initially they could include advice & guidance rather than formal tests for conformance.

Cheers,

-Alastair



--
John Foliot | Principal Accessibility Strategist

Deque Systems - Accessibility for Good

deque.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdeque.com%2F&data=02%7C01%7Cath%40siteimprove.com%7C4e415f9c527e4e84e30208d60e9d13a0%7Cad30e5bc301d40dba10a0e8d40abe0f9%7C0%7C0%7C636712465143574858&sdata=sgVbnGpC0ScavPm6jsyl1S7MdV1L3SsNc1cCyuGXFR0%3D&reserved=0>