We had an interesting discussion about result aggregation yesterday. Based
on that, what I want to do with this going forward is to make this part of
the benchmarking method instead. We need to measure the accuracy of rules,
this is part of our scope, and the only way I know to do this is to compare
results from a rule, to results from expert evaluations. That does mean we
need to write down how you get from rule results, to success criteria
results, but we can do that without defining a data format.
For the implementation of the benchmark we will obviously need some sort of
format. It will need to have data fed to it, so that has to fit a format,
and to me, EARL makes the most sense here. But this seems to me an
implementation detail of the benchmarking tool that is planned for year 3,
so as far as I'm concerned we don't have to put this into the framework.
So what i want to do is the following: I'll move the aggregation section
over to the benchmarking section, and I'll take out the JSON-LD examples,
replacing them with simpler, plain language examples instead.
Let me know if you have any additional suggestions!
--
*Wilco Fiers*
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
