Re: Scoring and Dashboards

Hi Wayne,

As I've not actually used each of these 11 tools, I cannot say for sure. I
simply did an image search for "Dashboard + [tool name]" and
collected/posted the results. (I already knew that most of these tools
offered that function, so I was searching for visual evidence of things I
already knew to exist).

I know that Deque's Enterprise tool (Comply) allows you to set which
version of WCAG you wish to report too (and also has other settings for
ACAA and Section 508 Trusted Tester if you choose)

[image: image.png]

The concern I have however is that in my experience, and with the evidence
I've provided, it suggests that site owners want to be able to monitor the
accessibility health of their sites ongoing - that they do NOT want
one-and-done testing and then "...*fingers crossed nothing breaks*."

As I've stated multiple times now, in this scenario (plausible, realistic,
and *actual*) old testing-results data diminishes in value over time (does
anyone disagree there?), and so I am seeking a scoring mechanism that
addresses that reality as well. There has been considerable focus on the
end user throughout this entire process, but they are not the only
stake-holders at the table: there are site owners, content creators,
legislators and other governing bodies, and more, who will all be impacted
by our decisions.

A system that results in *awesome* usability for individual users, but does
not work for content creators and site owners will not be taken up at
scale: whatever WCAG 3.x looks like will need to work for those
stakeholders as well.

JF

On Tue, May 12, 2020 at 2:16 PM Wayne Dick <wayneedick@gmail.com> wrote:

> Do these dashboards use full WCAG 2.most-current?
> I am finding a lot of ignoring of reflow. Is this covered?
> Best, Wayne
>
> On Mon, May 11, 2020 at 9:10 AM John Foliot <john.foliot@deque.com> wrote:
>
>> Hi All,
>>
>> During our calls last week, the use-case of monitoring conformance
>> dashboards was raised.
>>
>> One important need for *on-going score calculation* will be for usage in
>> these scenarios. After a bit of research, it appears that many different
>> accessibility conformance tools are today offering this
>> feature/functionality already.
>>
>> Please see:
>>
>>
>> https://docs.google.com/document/d/1PgmVS0s8_klxvV2ImZS1GRXHwUgKkoXQ1_y6RBMIZQw/edit?usp=sharing
>>
>> ...for examples that I was able to track down. (Note, some examples today
>> remain at the page level - for example Google Lighthouse - whereas other
>> tools are offering composite or aggregated views of 'sites' of at least
>> 'directories' [sic].)
>>
>> It is in scenarios like this that I question the 'depreciation' of
>> user-testing scores over time (in the same way that new cars depreciate
>> when you drive them off the lot, and continue to do so over the life of the
>> vehicle).
>>
>> Large organizations are going to want up-to-date dashboards, which
>> mechanical testing can facilitate quickly, but the more complex and
>> labor-intensive tests will be run infrequently over the life-cycle of a
>> site or web-content, and I assert that this infrequency will have an impact
>> on the 'score': user-test data that is 36 months old will likely be 'dated'
>> over that time-period, and in fact may no longer be accurate.
>>
>> Our scoring mechanism will need to address that situation.
>>
>> JF
>> --
>> *John Foliot* | Principal Accessibility Strategist | W3C AC
>> Representative
>> Deque Systems - Accessibility for Good
>> deque.com
>> "I made this so long because I did not have time to make it shorter." -
>> Pascal
>>
>>
>>
>>

-- 
*​John Foliot* | Principal Accessibility Strategist | W3C AC Representative
Deque Systems - Accessibility for Good
deque.com
"I made this so long because I did not have time to make it shorter." -
Pascal