Re: Rule result aggregation

Hey team,
I've updated the benchmarking section, and I've added rule aggregation to
this area, as from the discussion last week this seemed like the best way
forward. Please review and let me know what you think:

https://github.com/w3c/wcag-act/pull/47/files?diff=split

Wilco

On Fri, Feb 17, 2017 at 1:32 PM, Wilco Fiers <wilco.fiers@deque.com> wrote:

> We had an interesting discussion about result aggregation yesterday. Based
> on that, what I want to do with this going forward is to make this part of
> the benchmarking method instead. We need to measure the accuracy of rules,
> this is part of our scope, and the only way I know to do this is to compare
> results from a rule, to results from expert evaluations. That does mean we
> need to write down how you get from rule results, to success criteria
> results, but we can do that without defining a data format.
>
> For the implementation of the benchmark we will obviously need some sort
> of format. It will need to have data fed to it, so that has to fit a
> format, and to me, EARL makes the most sense here. But this seems to me an
> implementation detail of the benchmarking tool that is planned for year 3,
> so as far as I'm concerned we don't have to put this into the framework.
>
> So what i want to do is the following: I'll move the aggregation section
> over to the benchmarking section, and I'll take out the JSON-LD examples,
> replacing them with simpler, plain language examples instead.
>
> Let me know if you have any additional suggestions!
>
> --
> *Wilco Fiers*
> Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG
>



-- 
*Wilco Fiers*
Senior Accessibility Engineer - Co-facilitator WCAG-ACT - Chair Auto-WCAG