W3C home > Mailing lists > Public > public-wai-ert@w3.org > May 2005

Re: Confidence claims in EARL assertions

From: Nils Ulltveit-Moe <nils@u-moe.no>
Date: Wed, 25 May 2005 16:43:43 +0200
To: shadi@w3.org
Cc: public-wai-ert@w3.org
Message-Id: <1117032223.12864.358.camel@moe-ulltveit-moe.com>

Hi Shadi,

ons, 25,.05.2005 kl. 15.47 +0200, skrev Shadi Abou-Zahra:
> Remember, we are looking at the probability that a result is indeed precise, 
> not the impact or effect of that result in any further context. 
> For the specific case of Web accessibility, the relative priority level of the checkpoint 
> (or success criteria) indicates that impact for people with disabilities. This is different 
> from the earl:confidence value we are talking about here.

The precision of the result, or level of confidence may be interesting,
if seen from the perspective of an individual test.

However, I have a problem seeing how the conficence parameter can be
used in aggregation of test results.

One way of aggregating the WCAG checks, is to regard it as the web
accessibility equivalent of an "EU vehicle test" or MOT for
accessibility on web. In this case you can define success criteria as
all level A checks or AA or AAA passed. If it does not pass, EARL
describes the discrepancies. Aggregation could be specified as the
percentage of A, AA, AAA passed.  Policy makers could get nice graphs
showing how far they are from e.g. reaching level AAA, and if the trend
is increasing or decreasing, and possibly the speed. However, if these
results are uncertain, it is not clear for me how the end uncertainty
would be. Also, in the end, the question the user will pose is: Is this
web site accessible for me?

Another way of modelling the uncertainty, is to try to use an user
centric model, that tries to model the probability that a collection of
tests imposes an accessibility barrier for a given user. In such a
model, the user would eventually be able to get an indication whether
that page was accessible for him with a given level of confidence.
Aggregating data and modelling the accessibility barriers is maybe
easier in this model.

These are difficult questions, and neither strategy is completely right
or wrong.

What are your views on how data can be aggregated?

Best regards,
-- 
Nils Ulltveit-Moe <nils@u-moe.no>
Received on Wednesday, 25 May 2005 14:36:51 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 8 January 2008 14:18:25 GMT