AW: Additional Point/Question: problem centered / page centered evaluation

Hi TF,

what I have described as an probable testing procedure was not about 100%
Conformance. If all/some/someone understood this in that way, than it was a
misunderstanding. I've already pointed out, that it is especially when we
have to check huge pages not possible to check every single page. The
question is not for the 100%, not for checking - especially on very huge
pages every SC on every single page (probably thousands). What I've
described might also not be *the perfect test*. Of course we have to
consider and discuss pragmatism, financial things, time and all that (I've
tested enough websites in the last seven years to know that). And we also
have to pay attention on the Conformance Requirements of WCAG 2.0 and on the
three criteria for the quality of tests in general: Reliability, Objectivity
and Validity. We are in between those points and all are important. Some are
must-haves, some should-haves and I think we will find also some
nice-to-haves.

I started to work in the field of web accessibility in 2002. Since 2005 I
work as Accessibility Consultant. For now nearly seven years I'm consulting
accessibility, give trainings and workshops, writing about it and tested a
lot of websites. More and more I also work in the field of project
management. All those things most of us I here in this TF are doing. In the
first three years I checked pages according a test method where the result
depends upon degrees and selected pages. For now 3,5 years I prefer - when
testing WCAG 2.0 - as described very very brief in this long mail about this
issue. 
 
My experiences with tests which are following the idea of points
(percentages I think is similar) are that there is a high risk that clients
are "thinking in points": "What do we have to do to get those 90 points?"
Yes, there are website owners and developers who have accessibility in mind
and we will have in future more and more. But we also have web site owners
who have points in mind, who are not (enough) looking for the SCs itself not
for the accessibility - especially when they don't want to make it but have
to. In a direct contact with a website owner, the developer,... a consultant
can speak about this. He/she can sensitize website owners and developers
about accessibility goals. But just in a direct consultant/client situation
and sometimes successfully but sometimes not. Denis spoke about obsession.
How can we avoid obsession in points/percentage? Like the following: "We
have some videos, without subtitles, but we are losing just 3 points (3,
just to mention a number). That's ok and probably the tester will not find
the pages with the videos. Then we are lucky and don't loose any points
because of that." This can conjugated not through the very end of every SC
but for a lot. Considerations like this are happening too often for calling
them individual cases and those considerations are also counter-productive
for the accessibility goal, especially for the users. We don't live in an
ideal world...


Best

Kerstin

> -----Ursprüngliche Nachricht-----
> Von: public-wai-evaltf-request@w3.org [mailto:public-wai-evaltf-
> request@w3.org] Im Auftrag von Denis Boudreau
> Gesendet: Freitag, 9. September 2011 15:43
> An: Eval TF
> Betreff: Re: Additional Point/Question: problem centered / page
> centered evaluation
> 
> Hi again,
> 
> On 2011-09-09, at 6:06 AM, Detlev Fischer wrote:
> 
> > I think we should give up on the idea of a 'perfect' conformance test
> covering *every page and state of a site* with the aim of checking for
> 100% conformance to WCAG 2.0. Looking back on many many tests we have
> carried out in the past - mostly sites that were actually designed with
> accessibility requirements in mind - there has not been a single
> perfect site.
> 
> +1 000 000 000 000 000 000 :)

> > If it was possible to devise perfect test and set the goal at 100%
> conformance, it would be meaningless because (almost) every site would
> fail. A perfect test may even lead to distorted evaluations: the method
> might encourage evaluators of pretty good sites to close their eyes on
> some remaining issues (there ALWAYS ARE SOME issues) in order to reach
> an impossibly ambitious conformance goal.
> 
> And it can also counter-productive because people might lose sight of
> the accessibility goal. When an organization is obsessed with
> conformance (because it's a law for example), then they forget that
> even though they might not *comply* with the each and every requirement
> of the standard, every little thing being done actually improves
> accessibility anyway.
> 
> Accessibility, as opposed to compliance, is rarely black or white.
> 
> 
> 
> --
> Denis Boudreau, président
> Coopérative AccessibilitéWeb
> 
> 1751 rue Richardson, bureau 6111
> Montréal (Qc), Canada H3K 1G6
> Téléphone : +1 877.315.5550
> Courriel : dboudreau@accessibiliteweb.com
> Web : www.accessibiliteweb.com

Received on Sunday, 11 September 2011 09:43:32 UTC