Re: Costs of testing with Silver

Thank you, Wilco, for expanding on the question of cost, which has helped me better understand the problem, but like John I’m having trouble squaring the circle of more affordable tests with no detriment to people with disabilities.

It got me wondering if there exists anywhere profiles and scenarios of representative *organisations* who are expected to use Silver, in addition to personas from the well-established Silver stakeholder groups—developers, designers, lawmakers etc? 

And if there aren’t, it sounds like it might be a good idea to create some (perhaps as part of James’ suggested shared doc), so we can better understand the characteristics and constraints of organisations like the ones we’re hearing have problems with WCAG 2, and who need to be better served by Silver. 

Then we’d be better placed to work out what’s needed to ensure Silver meets the needs of different types of organisations, without sacrificing accessibility for people with disabilities.

Dave

> On 5 Sep 2018, at 21:38, John Foliot <john.foliot@deque.com> wrote:
> 
> Hi Wilco,
> 
> I'm certainly not suggesting that you wish to "ignore the needs of certain people", only that a simplified (no cost/low cost) testing strategy may cause a false sense of "success", or somehow diminish the need for robust testing, which ultimately does leave more users behind. 
> 
> It's easy to argue that Site A at "70% accessible" is better than Site B at "35% accessible", and in a positive light that equals one site that is twice as accessible as another, but in either instance the sites are still inaccessible to between 30% and 65% of their potential users. I struggle with that, and I further struggle with us saying that 70% is good enough for smaller or "simpler" sites and shops that cannot (or will not) pony up to do the proper testing. To my thinking, half-baked testing is as lousy as no testing to the end user.
> 
> Looking at your bullet points:
> 
> > - Shift responsibilities over to UA and AT vendors, instead of content authors. For example have ATs solve the issue of single key shortcuts, and require browsers to always have a visible tab focus for keyboard users.
> 
> I fully support that idea, however *HOW* do we accomplish that? The W3C does not have a mandate or mechanism to force anyone to do anything, and so while the idea is right in principle, implementation remains, well, where we are today. 
> 
> Have I given up hope? No, but I am also pragmatic in thinking that vendors alone won't solve some of the problems, especially as the impacted user-group decreases in volume. Today, all browsers *DO* have native visible tab focus indication (with at least a few using an indication that fails contrast requirements), but thanks to author control, that visible tab focus can be "turned off", and I'll bet a year's salary that no browser will remove that ability any time soon.
> 
> Additionally, not all accessibility issues are hardware or software-based issues: often it is content being created by the site owner that is introducing the issues. We've all seen this multiple times, especially when authors are using CMS tools, where the templates may be "accessible", and yet the content author writes about clicking on the round red button on the right - another "easy" accessibility problem to point out, but I'm not sure that the kinds of no-cost/low-cost testing strategies such as you are proposing could capture that problem. (Given internationalization concerns, I struggle to think of any automated test that could be applied directly to authored content)
> 
> 
> > - Use more requirements such as parsing that look for bad practices known to lead to inaccessible content, instead of things that are directly inaccessible. Those things tend to be much easier to test (think "don't skip heading levels")
> 
> While I understand your point, the example you pointed out is only really an issue to screen reader users, as sighted users generally don't see the "level", but rather the outcome when that "level" has been styled. A heuristic test like that is still important, don't get me wrong, but it is but a sliver of the number of issues that can be found, and again relies on "structure" as opposed to "content", which at least 10 of the current WCAG 2.0 SC are dependent on successfully meeting today.
> 
> 
> > - Remove some of the complexities and exceptions in the entry level. For example the "media alternative for text". Maybe we have a "simple sites" version of Silver that doesn't have this stuff, but a "complex site" which allows them, but adds in stuff that's rarely a problem on simple sites.
> 
> Again, I understand the larger point, but with that approach we have to also declare, formally, the difference between a "simple" site and a "complex" site. 
> Is a one page site that features a professor's video lecture at a university "simple" or "complex"? Why?
> 
> That discussion could go either way: on the simple side it's a web page with a single heading, a video rendering area, and a footer that says "Copyright XYZ University". Yet because that single, simple page contains a video, it has an additional slew of requirements that cannot be accurately machine tested. 
> 
> Moving beyond that, say that the single page I just outlined was one of a total of 5 pages created by the Professor, and all of the 5 pages use the same basic template (header, nav, main, footer). It would be very easy to claim it's a "simple" site (because 80% of it is), but the remaining 20% (the video page) is "complex". Now what? 
> 
> If four of the 5 pages "pass", yet the video page is an abject failure, I can see the site owner arguing "ya, but I got an 80% pass score" but the impacted blind, deaf, or deaf/blind users are still left with... potentially nothing. This is the concern - that, and defining what is "simple" versus what is "complex", which is another subjective call at a higher level.
> 
> 
> > - Remove ambiguities and technology agnostic language from definitions. Even if we do it just for HTML and SVG. If we can make it unambiguous, that's even better because that means it's automatable.
> 
> +1 to removing ambiguity, although I am less sure about removing technology agnostic language from the definitions or Success Criteria. 
> Since the beginning, WCAG 2.0 has mostly been about functional outcomes, and not methods of meeting a specific SC (which is one of the reasons why Techniques are both non-normative, and outside of the normative spec), and so keeping the ideas open and broad, without specific techniques, is a goal I would continue to support.
> 
> 
> > All of these things help reduce the costs of accessibility testing without impacting PwD in the slightest.
> 
> I'm not sure I understand what you mean here: if the tests *aren't* impacting PwD (in a positive way), why are we testing? To prove we did a test? (That again sounds like check-box accessibility, WCAG 1.0-style)
> 
> Don't get me wrong, the more we can do to make 'testing' easier, faster, cheaper, and more accurate is a goal I fully support. My concern arises when we start to suggest that "accessibility testing" is but a button click away, and that needing to *think* about accessibility is less important than using patterns that match accessibility techniques. That's because, as experienced professionals, we all know that half the time, the answer to any given accessibility question usually starts out with, "That depends..."
> 
> I recognize that cost is a factor, I really do. But to my mind, quality must supersede cost if we truly want to benefit the target audience of PwD. Focusing on the lowest common denominator (make it cheap) also has a cost - to the end user. It's not a trade-off I personally am comfortable with.
> 
> JF
> 
> 
> On Wed, Sep 5, 2018 at 1:01 PM, Wilco Fiers <wilco.fiers@deque.com> wrote:
> Hi John, Charles,
> Neither of you have responded to the actual concerns that I've raised that, we should be concerned in making accessibility testing affordable for smaller organisations with no legal requirement to follow accessibility standards.
> 
> I am not suggesting that we ignore the needs of PwD. Making this out as though I've suggested that we ignore the needs of certain people is misrepresenting my concern. There are a lot of ways we can approach this concern of cost that wouldn't impact PwD in the slightest. Just to list a few:
> 
> - Shift responsibilities over to UA and AT vendors, instead of content authors. For example have ATs solve the issue of single key shortcuts, and require browsers to always have a visible tab focus for keyboard users.
> - Use more requirements such as parsing that look for bad practices known to lead to inaccessible content, instead of things that are directly inaccessible. Those things tend to be much easier to test (think "don't skip heading levels")
> - Remove some of the complexities and exceptions in the entry level. For example the "media alternative for text". Maybe we have a "simple sites" version of Silver that doesn't have this stuff, but a "complex site" which allows them, but adds in stuff that's rarely a problem on simple sites.
> - Remove ambiguities and technology agnostic language from definitions. Even if we do it just for HTML and SVG. If we can make it unambiguous, that's even better because that means it's automatable.
> 
> All of these things help reduce the costs of accessibility testing without impacting PwD in the slightest. Now I'm not saying we shouldn't shift some things around. Perhaps we can pick apart existing requirements such as those for info and relationships and keep the essential part in the entry level, but move the less common / less burdensome issues to the "Silver / AA" level, or to the requirement for "complex sites" whatever that ends up being.
> 
> Again, I'm not asking that we ignore certain people's problems. I'm asking that we think about lowering the cost. We can do that without impacting PwD at all, if we so choose, or we can reprioritise a few things. We'll have that discussion when it gets to that point. What I'm asking is that you take a stance on what you think about making accessibility testing with Silver affordable for small websites.
> 
> Wilco
> 
> 
> 
> On Wed, Sep 5, 2018 at 5:06 PM John Foliot <john.foliot@deque.com> wrote:
> Supporting legislation and policy is not the purpose of accessibility guidelines. People are.
> 
> Amen Charles, except...
> 
> No matter where we collectively "draw the line", it's going to fall somewhere at less than 100% accessible - that's just a reality we need to accept: accessibility is a long-tail problem that ends at the individual, and no standard or guideline will include every user on earth. That's simply reality colliding with pragmatism.
> 
> No matter the model, once legislations grab hold of the model there will be a bright and shiny line between "pass" and "fail", even though collectively we know that accessibility is far more nuanced than that. 
> 
> My concern is that a "simple" accessibility test (i.e., low cost/no cost) will be a bit of a paper dragon: checking of tick boxes in pursuit of some kind of truncated solution just feels wrong to me, as entities that choose that route aren't really concerned about PwD, they are instead concerned about not being sued. Reaching for the lowest common denominator just feels wrong, as it positions web accessibility not for its benefits, but rather its liabilities.
> 
> I fully support free resources that help developers and content creators "do better", and automated tests and other heuristic checks have their place, but to create a "benchmark" based simply on free automated tests is going to fail PwD as often as it benefits, with the added 'burden' of falsely suggesting that because the "ACME Widget Web Site" passed all the automated tests, it's "blessed" as somehow being accessible.
> 
> The problem isn't testing, its education.
> 
> I've long held that one of the problems with WCAG today is that most people focus almost exclusively on the actual SC, and not on the Principles and Guidelines. A renewed focus on the understanding piece, as opposed to measuring compliance to specific checkpoints, is to me the more critical piece. Silver should be focused on that, and it appears to be in many ways (the desire/need for more "human readable" content is, I argue, an example of the direct backlash we see/feel on focusing on the SC's and not the reason behind the SC's, which results in folks struggling to decipher the text of the SC's and the perception that this is "hard"). 
> 
> Can individual checkpoints be created in such a way that auto or programmatic testing can speed progress? I would hope so, and would support that goal. 
> 
> But to establish a metric that suggested that a certain level of compliance could be "claimed" based solely on auto-testing would be something I would strenuously protest as being far too little, and feels very retrograde to me: we moved off of WCAG 1.0 for solid reasons (including the very fact that you could meet each of the WCAG 1.0 "tick-boxes" and still have a very inaccessible web page/site). Returning to a model that repeats some of the problems from the past is going backward IMHO, and I continue to fail to see how it benefits anyone except entities and sites that want to appear to be doing something, yet aren't prepared to do the hard miles required. It's like saying "I want to see all the iconic stops along US Route 66, but I don't want to spend all that time driving..." It's completely missing the bigger point.
> 
> JF
> 
> On Wed, Sep 5, 2018 at 8:27 AM, Hall, Charles (DET-MRM) <Charles.Hall@mrm-mccann.com> wrote:
> I wonder if we can restart this conversation with a mission or goal in mind.
> 
>  
> 
> Instead of discussing cost of procurement, cost of testing, cost of remediation or any other associated cost generally associated with a business, can we acknowledge the cost to people without access?
> 
> Accessibility guidelines are a method to aid in ensuring people can access the web.
> 
>  
> 
> The entire and only two reasons that any public policy or legislation (anywhere in the world) has ever specifically cited the WCAG x.x is because it was the most comprehensive guidance currently available at the time the legislation was considered AND that guidance was published by the W3C as a recommendation which is considered a web standard.
> 
>  
> 
> The only reason that any said legislation cites a compliance or conformance requirement is because a strict and explicit conformance is part of WCAG.
> 
>  
> 
> Any new guidance that is also comprehensive and a recommendation (standard) would supersede the previous guidance when drafting any new policy or legislation. The only constant is change.
> 
>  
> 
> Supporting legislation and policy is not the purpose of accessibility guidelines. People are.
> 
>  
> 
> If starting from scratch, but using everything we have collectively learned from research and experience:
> 
>  
> 
> Can we create new guidelines that support (more) people and solve all (or most of) the problems that research revealed within the current guidelines?
> 
>  
> 
> Can we create a conformance model that is not strict and explicit to technical details, but is to human factors?
> 
>  
> 
> Can we do that while also considering the impact on (all) stakeholders?
> 
>  
> 
>  
> 
> Charles Hall // UX Architect, Technology
> 
>  
> 
> charles.hall@mrm-mccann.com
> 
> w 248.203.8723
> 
> m 248.225.8179
> 
> 360 W Maple Ave, Birmingham MI 48009 
> 
> mrm-mccann.com
> 
>  
> 
> <image001.jpg>
> 
> Relationship Is Our Middle Name
> 
>  
> 
> Ad Age B-to-B Agency of the Year, 2018
> 
> Ad Age Agency A-List 2016, 2017
> 
> Ad Age Creativity Innovators 2016, 2017
> 
> North American Agency of the Year, Cannes 2016
> 
> Leader in Gartner Magic Quadrant, 2017, 2018
> 
>  
> 
>  
> 
>  
> 
> From: Mark Tanner <mktanner@btinternet.com>
> Date: Wednesday, September 5, 2018 at 7:49 AM
> To: Silver Task Force <public-silver@w3.org>
> Subject: Re: [EXTERNAL] Costs of testing with Silver
> Resent-From: Silver Task Force <public-silver@w3.org>
> Resent-Date: Wednesday, September 5, 2018 at 7:48 AM
> 
>  
> 
> I am writing to support and slightly expand upon the points made by John. WCAG 2.0 is currently referenced throughout the EU as a key component of digital accessibility legislation.
> 
> As John pointed out, the EU standard EN 301 549 lays out "Accessibility requirements suitable for public procurement of ICT products and services in Europe" http://mandate376.standards.eu/standard
> 
> This standard not only references WCAG 2.0 as normative, but specifically  details all the WCAG 2.0 success criteria at reasonable lengthhttp://mandate376.standards.eu/standard/technical-requirements/#9
> 
> However, as from 23 September, 2018, the standard has a wider application beyond procurement.  EU DIRECTIVE  2016/2102 on digital accessibilityhttps://eur-lex.europa.eu/eli/dir/2016/2102/oj , which comes into legal effect in all EU countries (including the UK) requires that:
> 
> ·       Public sector websites created after 23 September 2018 need to comply with the accessibility requirements from 23 September 2019
> 
> ·       Public sector websites created before 23 September 2018 need to comply with the accessibility requirements from 23 September 2020
> 
> ·       Public sector mobile applications  need to comply with the accessibility  requirements from 23 June 2021.
> 
> Paragraphs 36 and 37 of EU DIRECTIVE  2016/2102 are clear:
> 
> “36. The accessibility requirements set out in this Directive … describe what must be achieved in order for the user to be able to perceive, operate, interpret and understand a website, a mobile application and related content …
> 
> “37 … Those principles of accessibility are translated into testable success criteria, such as those forming the basis of the European standard EN 301 549 V1.1.2  ….Pending publication of the references to harmonised standards, or of parts thereof, in the Official Journal of the European Union, the relevant clauses of European standard EN 301 549 V1.1.2 (2015-04) should be considered as the minimum means of putting those principles into practice.”
> 
> As EN 301 549 currently uses WCAG 2.0 as its normative reference for all public sector websites this means that all public sector websites and applications throughout all EU countries will need to meet WCAG 2.0.
> 
> It is also important to note that EN 301 549 is currently being updated to map to WCAG 2.1 to level AA). See Explanatory Memorandum To The Public Sector Bodies (Websites And Mobile Applications) Accessibility Regulations 2018 No. 852 http://www.legislation.gov.uk/uksi/2018/852/pdfs/uksiem_20180852_en.pdf
> 
> All this clearly has important implications for any shift in the future from WCAG to Silver.
> 
> Mark Tanner
> 
> This message contains information which may be confidential and privileged. Unless you are the intended recipient (or authorized to receive this message for the intended recipient), you may not use, copy, disseminate or disclose to anyone the message or any information contained in the message.  If you have received the message in error, please advise the sender by reply e-mail, and delete the message.  Thank you very much.
> 
> 
> 
> 
> -- 
> John Foliot | Principal Accessibility Strategist
> Deque Systems - Accessibility for Good
> 
> deque.com
> 
> 
> 
> 
> -- 
> Wilco Fiers
> Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
> 
> 
> 
> 
> -- 
> John Foliot | Principal Accessibility Strategist
> Deque Systems - Accessibility for Good
> 
> deque.com
> 

--
David Sloan
--
UX Research Lead
The Paciello Group
https://www.paciellogroup.com
A VFO™ Company http://www.vfo-group.com/
--
This message is intended to be confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, please delete this message from your system and notify us immediately.
Any disclosure, copying, distribution or action taken or omitted to be taken by an unintended recipient in reliance on this message is prohibited and may be unlawful.

Received on Thursday, 6 September 2018 09:25:46 UTC