Considering existing AI/Machine-Learning metrics for assessing value quality under WCAG 3.0...

Hi, 

[Forwarding for FYI]

In the field of AI/Machine-Learning one of the most important things to set in a project is a measurable success metric.

Over the past few years more and more work has being done to define measurable success metrics In AI/ML, some of which might be very useful to consider when looking at better defining WCAG 3.0 Outcomes.

I will use Alternative Text as an example.  Simply stating that alternative text provided by a system be “descriptive” is not sufficient for AI/Machine-Learning projects - which want to clearly indicate their improvements. So over the past few years some seemingly good work has been done on defining metrics that measure the quality of generated captions by analyzing their semantic content.  See SPICE - Semantic Propositional Image Caption Evaluation (https://panderson.me/images/SPICE.pdf <https://panderson.me/images/SPICE.pdf>) as an example.  SPICE would be capable of testing human created alternative text for an image; even though it has been designed to test dynamically generated text alternatives.

With such an example in mind, might the AGWG be open to considering the use of such metrics, when available and appropriate of course, in their future development of testable WCAG 3.0 Outcomes?

Raised as a point of interest really, as I am seeming such metrics through my research work.

Very best regards

Alistair 

Alistair Garrison
CEO / Founder Accesseo Limited
((Newly) Invited Expert - AGWG)