Re: Seeking Automated WCAG Testing Tool with Quick Client Report Sharing

Hey all,
Thank you for your insights. My sincere apologies if I didn't clarify the
rationale behind requesting this type of tool.

I deeply value the thoroughness of manual auditing. However, the reason for
considering this tool is purely for client pitch purposes, offering a
glimpse into their accessibility challenges and an opportunity to assist
further . It's crucial to emphasize that while this tool aids in initial
assessments, it never diminishes the significance of manual audits and
better experience for all, which I hold in the highest regard

Thanks again,
Kiran

On Tue, Feb 6, 2024 at 5:09 PM Steve Green <steve.green@testpartners.co.uk>
wrote:

> I think most professional testing companies use a similar approach, and
> since 2018 it has been mandated for UK central government departments, so
> lots of internal teams and freelancers are being exposed to it.
>
> However, I don't know of any national law in any country that mandates or
> even suggests such an approach. At most they simply require WCAG
> conformance, so it's no surprise that organisations see that as being the
> target.
>
> Steve
>
>
> -----Original Message-----
> From: Karen Lewellen <klewellen@shellworld.net>
> Sent: Tuesday, February 6, 2024 10:55 PM
> To: Steve Green <steve.green@testpartners.co.uk>
> Cc: Kiran <kiranph@gmail.com>; w3c WAI List <w3c-wai-ig@w3.org>
> Subject: RE: Seeking Automated WCAG Testing Tool with Quick Client Report
> Sharing
>
> Steve,
> unfortunately many companies do not find people like you!
> Instead they google a solution, have someone pretend to be disabled for 15
> minutes, and think they are done..I am looking at you Toronto public
> library.
> So, a person seeks inclusion, and they get told to match the testing.
> Your method is like building a house, careful proper testing so that once
> built  party guests can simply enjoy the food.
> However, both those making testing decisions, and the  end users are still
> miles apart where basic  public understanding and relations are concerned.
> Sadly there is only one of you smiles.
> Kare
>
>
> On Tue, 6 Feb 2024, Steve Green wrote:
>
> > If a company chooses to rely entirely on automated tools, they have been
> badly advised. However, automated testing is often an important aspect of a
> more comprehensive testing approach, so don't write it off.
> >
> >
> >
> > User testing is valuable, but it's not possible to do comprehensive user
> testing on anything but the smallest websites. We need an approach that
> scales. Our approach (and I suspect that of many others) is:
> >
> >
> >
> >  1.  Do a manual audit of a representative subset of pages, then fix the
> non-conformances and verify the fixes. If you don’t do this first, the
> automated testing will produce unmanageably large reports.
> >
> >  2.  Do an automated WCAG test. This will find lots more issues because
> the “representative subset of pages” used for manual testing almost never
> contains absolutely everything. Fix the non-conformances and verify the
> fixes.
> >
> >  3.  Test the representative subset of pages with a range of assistive
> technologies, fix the issues and verify the fixes.
> >
> >  4.  Only now would we do user testing with disabled participants. If
> you omit any of the previous stages:
> >
> >     *   This stage will find issues you could have found earlier and far
> cheaper.
> >
> >     *   You won’t find some issues that should have been findable
> because they are masked by other issues that could have been removed.
> >
> >     *   There will be so many minor issues (that could have been
> removed) that participants may adopt a negative attitude to the website and
> the testing process.
> >
> >     *   When you encounter an issue that could only be found at this
> stage, diagnosis may be hampered by the presence of other issues that could
> have been removed earlier.
> >
> >
> >
> > Each stage builds on the ones before and finds issues that previous
> stages could not have found.
> >
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: Karen Lewellen <klewellen@shellworld.net>
> > Sent: Tuesday, February 6, 2024 9:54 PM
> > To: Steve Green <steve.green@testpartners.co.uk>
> > Cc: Kiran <kiranph@gmail.com>; w3c WAI List <w3c-wai-ig@w3.org>
> > Subject: RE: Seeking Automated WCAG Testing Tool with Quick Client
> > Report Sharing
> >
> >
> >
> > Speaking personally, and respecting why they exist..after a fashion, I
> want to support Steve's point but for a different reason.
> >
> > In many ways these automated tools teach  those outside of the
> experience that living with a disability is uniform, that all those sharing
> a label are  interchangeable, and that if the test says its fine, then the
> problem may  be with the person living with that disability experience.
> >
> > as in, if you were just disabled the way our test defined things, use
> those tools,  etc., your lack of access would go away.
> >
> > Fully owning that my  reading is not likely as broad as many here, far
> too often some disability populations get left out..because they do not use
> a screen reader.
> >
> > If the automated testing tool focuses on this, and not say navigating
> via voice, what are you teaching the company who chooses to rely entirely
> on automated tools?
> >
> > Hey, if I use this, I never have to actually  stand in a room with a
> disabled  person!  A computer can mimic their lies and individuality just
> fine, no human understanding necessary.
> >
> > I wish I were kidding, but I speak here from some experience.
> >
> > again speaking personally,I find the idea of simulated testing quite
> repulsive, unless absolutely paired with manual work done by humans..along
> with a healthy dose of, not everyone will fit in these boxes.  use
> progressive enhancement design instead of pretending to have a disability.
> >
> > Just my take,
> >
> > Karen
> >
> >
> >
> >
> >
> >
> >
> > On Tue, 6 Feb 2024, Steve Green wrote:
> >
> >
> >
> >> The desktop version of SortSite can export an HTML version of its
> report, and it would be easy to add a logo to that. Although the HTML
> version looks the same as the report in the tool, it lacks the ability to
> drill down into the source code. This may not matter for your client.
> >
> >>
> >
> >> However, I would advise against sending the raw results from any
> automated testing tool. Our experience is that many of the results cannot
> be taken at face value. Issues include:
> >
> >>
> >
> >>
> >
> >>  *   Tools report false positives due to bugs in the tool.
> >
> >>  *   Tools report false positives due to the use of heuristics that
> don’t always give the correct result.
> >
> >>  *   Some false positives can cause multiple knock-on errors.
> >
> >>  *   Tools identify real faults, but diagnose and report them
> incorrectly.
> >
> >>  *   Tools identify real faults and diagnose and report them correctly,
> but recommend the wrong remedial action.
> >
> >>  *   Tools identify real faults that have no effect on the user
> experience and can be ignored.
> >
> >>
> >
> >> These raw reports are of little value to clients. The value you provide
> to your client is in the analysis of the raw reports and provision of
> corrected results and recommendations.
> >
> >>
> >
> >> We make an exception for our US clients because our advice to them is
> different from non-US clients. Due to the high prevalence of so-called
> drive-by law suits from ambulance chasing lawyers who use automated testing
> tools to identify potential targets, we recommend fixing all issues
> including false positives so automated tools don’t find any issues at all.
> In such cases, there is value in sending the raw report to the client.
> >
> >>
> >
> >> Steve Green
> >
> >> Managing Director
> >
> >> Test Partners Ltd
> >
> >>
> >
> >>
> >
> >> From: Kiran <kiranph@gmail.com<mailto:kiranph@gmail.com>>
> >
> >> Sent: Tuesday, February 6, 2024 3:40 PM
> >
> >> To: w3c WAI List <w3c-wai-ig@w3.org<mailto:w3c-wai-ig@w3.org>>
> >
> >> Subject: Seeking Automated WCAG Testing Tool with Quick Client Report
> >
> >> Sharing
> >
> >>
> >
> >> Hey all,
> >
> >>
> >
> >> I understand the importance of manual testing for thorough
> accessibility assessment. However, I am seeking an automated WCAG testing
> tool that facilitates quick sharing of reports with clients for swift
> review, featuring our company logo for branding consistency.
> >
> >>
> >
> >> Are you aware of any such tool/platform I can use at a reasonable price?
> >
> >>
> >
> >> Thanks!
> >
> >> ~ Kiran
> >
> >>
> >
>