RE: Another comment about confidence value.

Hi Paul,

tir, 19,.04.2005 kl. 10.55 +0100, skrev Paul Walsh:
> Sorry, my answer below was a bit short, that was no intentional as I
> decided to answer the email as I was rushing to a short meeting. I
> appreciate the magnitude of the testing you do. What I'm trying to say,
> is that an independent validator who provides a branded stamp of
> approval should not automate the majority of their testing - not in the
> UK anyway where it appears to be more open to litigation than most
> countries. My comment was not directed at you Nils :)

That is OK.

We are working on a research vehicle for large scale assessments. Time
will tell how successful we are. We are not going to provide
certifications or quality marks in our project. We aim at being able to
measure trends over time, and also being able to focus on specific areas
and performing targeted assessments over a large number of sites, and
being able to compare accessibility and assessment methods in different
geographical locations. What we need is sufficient precision for such
measurements.

> We are doing quite different measurements. We will be trying to do
> automatic assessments of a large number of sites (several thousand)
> regularly. We will need to do some manual testing, and will base our
> tests largely on automatic assessments. In our case we need to base
> ourself on probability theory and best practices in statistics to reach
> numbers that approximate the perceived accessibility over a large number
> of assessments, to make it feasible.
> [PW] This will not be accurate and I would question the process itself
> of using automation for the majority of your validation.

I know this is a problem. I have dealt with similar problems in my
previous job, which dealt with manual versus automatic categorisation of
alarms from intrusion detection systems. In that case we were able to
improve efficiency and precision with a combination of manual and
automatic assessments. However, these are issues we are doing research
on, so I cannot tell in advance how well such an approach would work for
accessibility assessments.

Mvh.
-- 
Nils Ulltveit-Moe <nils@u-moe.no>

Received on Tuesday, 19 April 2005 11:21:13 UTC