Re: After today's call (Comments to John Foliot's alternative scoring proposal)

On 12/08/2021 23:43, John Foliot wrote:
> And yet, this is the exact problem we have today: evaluators attempting 
> to subjectively decide (for other users) whether a textual alternative 
> is "good enough". Says who?

It's a problem today, and will be a problem tomorrow, because of the 
very nature of what it is trying to assess. The only way out of this 
seems to be to ignore the problem altogether and just test the 
mechanistic aspect ("if element has alt attribute, pass, otherwise 
fail", sprinkled with some AI heuristics?) and call it a day instead, 
with a soft "above and beyond, here's some protocols/guides for the 
stuff that's not covered directly here".

Admittedly, I'd quite like to be able to just outsource the boring 
"normative pass/fail" to a machine in the end and concentrate on 
education/improvement once a client gets past that initial "we just want 
to tick a box that says WCAG compliant" (though without the threat of 
litigation I wonder how many clients we'd be left with at that point ;) ).

[...]

The idea of the protocols seems to simply separate out subjectivity, but 
fundamentally is it still there? If a site gets extra points for stating 
that they will follow a protocol, then somebody (auditor, judge, ...) 
still then needs to go and apply their subjective view on whether their 
extra point is deserved or not. If a final total score still depends on 
summing the unambiguous unit test scores and the extra points, 
subjectivity will still end up influencing the final score - and we'll 
still see auditors disagreeing on whether a particular extra point is 
deserved or not, unless only the pure *promise* of following a 
particular protocol counts, rather than its use/implementation. Or 
unless legally only the unambiguous unit test scores will count, at 
which point why would a client bother with the extra points (other than 
kudos)?

> I do understand that this is somewhat shocking to many, but I personally 
> cannot see another path forward, and it is clear from the first round of 
> feedback that industry are very unhappy with the amount of additional 
> subjectivity in our current draft, so I assert we do need to come up 
> with something different.

Well yes, the crux of the issue is of course the conundrum of 
"guidelines" (in the sense of recommendations/best practices, not 
absolute edicts - thinking for instance of Apple's Human Interface 
Guidelines which even Apple themselves sometimes knowingly break) having 
been bent, for better or worse, into unambiguous "rules"/laws - turning 
them from something that can form the basis for improvement/discussions, 
a framing tool, to something that can potential lead to fines and law 
suits, sadly. That's been the fundamental tension. (Along the lines of 
the old Bill Hicks sketch about the supreme court defining what 
"pornography" is...)

To be clear, the effort to reframe/try to tackle this problem is 
admirable, so not a sleight on you John. Just me airing my perennial 
frustration on how we've been trying for years, and still are, and will 
be long after I retire, to fit a square peg into a round hole, due to 
the very nature of what it is we're trying to define.

P
-- 
Patrick H. Lauke

https://www.splintered.co.uk/ | https://github.com/patrickhlauke
https://flickr.com/photos/redux/ | https://www.deviantart.com/redux
twitter: @patrick_h_lauke | skype: patrick_h_lauke

Received on Thursday, 12 August 2021 23:55:25 UTC