RE: Seeking Automated WCAG Testing Tool with Quick Client Report Sharing

I think most professional testing companies use a similar approach, and since 2018 it has been mandated for UK central government departments, so lots of internal teams and freelancers are being exposed to it.

However, I don't know of any national law in any country that mandates or even suggests such an approach. At most they simply require WCAG conformance, so it's no surprise that organisations see that as being the target.

Steve
 

-----Original Message-----
From: Karen Lewellen <klewellen@shellworld.net> 
Sent: Tuesday, February 6, 2024 10:55 PM
To: Steve Green <steve.green@testpartners.co.uk>
Cc: Kiran <kiranph@gmail.com>; w3c WAI List <w3c-wai-ig@w3.org>
Subject: RE: Seeking Automated WCAG Testing Tool with Quick Client Report Sharing

Steve,
unfortunately many companies do not find people like you!
Instead they google a solution, have someone pretend to be disabled for 15 minutes, and think they are done..I am looking at you Toronto public library.
So, a person seeks inclusion, and they get told to match the testing.
Your method is like building a house, careful proper testing so that once built  party guests can simply enjoy the food.
However, both those making testing decisions, and the  end users are still miles apart where basic  public understanding and relations are concerned.
Sadly there is only one of you smiles.
Kare


On Tue, 6 Feb 2024, Steve Green wrote:

> If a company chooses to rely entirely on automated tools, they have been badly advised. However, automated testing is often an important aspect of a more comprehensive testing approach, so don't write it off.
>
>
>
> User testing is valuable, but it's not possible to do comprehensive user testing on anything but the smallest websites. We need an approach that scales. Our approach (and I suspect that of many others) is:
>
>
>
>  1.  Do a manual audit of a representative subset of pages, then fix the non-conformances and verify the fixes. If you don’t do this first, the automated testing will produce unmanageably large reports.
>
>  2.  Do an automated WCAG test. This will find lots more issues because the “representative subset of pages” used for manual testing almost never contains absolutely everything. Fix the non-conformances and verify the fixes.
>
>  3.  Test the representative subset of pages with a range of assistive technologies, fix the issues and verify the fixes.
>
>  4.  Only now would we do user testing with disabled participants. If you omit any of the previous stages:
>
>     *   This stage will find issues you could have found earlier and far cheaper.
>
>     *   You won’t find some issues that should have been findable because they are masked by other issues that could have been removed.
>
>     *   There will be so many minor issues (that could have been removed) that participants may adopt a negative attitude to the website and the testing process.
>
>     *   When you encounter an issue that could only be found at this stage, diagnosis may be hampered by the presence of other issues that could have been removed earlier.
>
>
>
> Each stage builds on the ones before and finds issues that previous stages could not have found.
>
>
>
> Steve
>
>
>
>
>
> -----Original Message-----
> From: Karen Lewellen <klewellen@shellworld.net>
> Sent: Tuesday, February 6, 2024 9:54 PM
> To: Steve Green <steve.green@testpartners.co.uk>
> Cc: Kiran <kiranph@gmail.com>; w3c WAI List <w3c-wai-ig@w3.org>
> Subject: RE: Seeking Automated WCAG Testing Tool with Quick Client 
> Report Sharing
>
>
>
> Speaking personally, and respecting why they exist..after a fashion, I want to support Steve's point but for a different reason.
>
> In many ways these automated tools teach  those outside of the experience that living with a disability is uniform, that all those sharing a label are  interchangeable, and that if the test says its fine, then the problem may  be with the person living with that disability experience.
>
> as in, if you were just disabled the way our test defined things, use those tools,  etc., your lack of access would go away.
>
> Fully owning that my  reading is not likely as broad as many here, far too often some disability populations get left out..because they do not use a screen reader.
>
> If the automated testing tool focuses on this, and not say navigating via voice, what are you teaching the company who chooses to rely entirely on automated tools?
>
> Hey, if I use this, I never have to actually  stand in a room with a disabled  person!  A computer can mimic their lies and individuality just fine, no human understanding necessary.
>
> I wish I were kidding, but I speak here from some experience.
>
> again speaking personally,I find the idea of simulated testing quite repulsive, unless absolutely paired with manual work done by humans..along with a healthy dose of, not everyone will fit in these boxes.  use progressive enhancement design instead of pretending to have a disability.
>
> Just my take,
>
> Karen
>
>
>
>
>
>
>
> On Tue, 6 Feb 2024, Steve Green wrote:
>
>
>
>> The desktop version of SortSite can export an HTML version of its report, and it would be easy to add a logo to that. Although the HTML version looks the same as the report in the tool, it lacks the ability to drill down into the source code. This may not matter for your client.
>
>>
>
>> However, I would advise against sending the raw results from any automated testing tool. Our experience is that many of the results cannot be taken at face value. Issues include:
>
>>
>
>>
>
>>  *   Tools report false positives due to bugs in the tool.
>
>>  *   Tools report false positives due to the use of heuristics that don’t always give the correct result.
>
>>  *   Some false positives can cause multiple knock-on errors.
>
>>  *   Tools identify real faults, but diagnose and report them incorrectly.
>
>>  *   Tools identify real faults and diagnose and report them correctly, but recommend the wrong remedial action.
>
>>  *   Tools identify real faults that have no effect on the user experience and can be ignored.
>
>>
>
>> These raw reports are of little value to clients. The value you provide to your client is in the analysis of the raw reports and provision of corrected results and recommendations.
>
>>
>
>> We make an exception for our US clients because our advice to them is different from non-US clients. Due to the high prevalence of so-called drive-by law suits from ambulance chasing lawyers who use automated testing tools to identify potential targets, we recommend fixing all issues including false positives so automated tools don’t find any issues at all. In such cases, there is value in sending the raw report to the client.
>
>>
>
>> Steve Green
>
>> Managing Director
>
>> Test Partners Ltd
>
>>
>
>>
>
>> From: Kiran <kiranph@gmail.com<mailto:kiranph@gmail.com>>
>
>> Sent: Tuesday, February 6, 2024 3:40 PM
>
>> To: w3c WAI List <w3c-wai-ig@w3.org<mailto:w3c-wai-ig@w3.org>>
>
>> Subject: Seeking Automated WCAG Testing Tool with Quick Client Report
>
>> Sharing
>
>>
>
>> Hey all,
>
>>
>
>> I understand the importance of manual testing for thorough accessibility assessment. However, I am seeking an automated WCAG testing tool that facilitates quick sharing of reports with clients for swift review, featuring our company logo for branding consistency.
>
>>
>
>> Are you aware of any such tool/platform I can use at a reasonable price?
>
>>
>
>> Thanks!
>
>> ~ Kiran
>
>>
>

Received on Tuesday, 6 February 2024 23:09:28 UTC