Re: tools, testers, conformance, etc

An important question...

In this case I use the yardstick of good taste - "agrees with me" 
<grin/>. This is somewhat moderated by actual experience - my yardstick 
is a composite of experience with things that cause problems for some 
members of a wide variety of users. In general, the tests I have seen 
with these problems have been examined by a fairly narrow range of 
users.

I agree with Joe - testing with real users provides important reality 
checks - especially of the guidelines being used (this is also what 
Tina said). But the average group of testers is (in my experience) two 
or three, which is nowhere near enough.

Some examples:

It is often claimed on the WCAG list that there is no need in the real 
world for default text in form fields. I searched myself for some time 
to find out if anyone still needed this. It turned out that there are 
in fact a number of Macintosh screen-reader users, and the most used 
product (Outspoken - very high market share in a small market because 
it is the only "real" screenreader for Mac OS at the moment) requires 
this to get around a bug. I discovered this through Sidar's user 
testing, because one of our testers is a blind person who uses a 
Macintosh.

Similarly, I have worked with people who find flashing content 
distracting to the point of making content very difficult to access. 
(This isn't so rare in its milder form - take a group of people to a 
bar with a TV and there is often one person who cannot follow a 
conversation if they can see the TV.) Yet disability organisations who 
do their own reasonably comprehensive testing use flashing images on 
their own sites - apparently their testing group didn't have a problem.

There is almost always mileage on these lines in the use of language. 
Checkpoint 14.1 in WCAG 1 is important for many people (see for example 
the comments of RNID on WCAG2, to reduce arguments about people with 
cognitive disabilities). Having worked for some time on the question of 
simplifying language, I can often see ways it can be done (although 
often I can only see that it needs to be done, and if I don't have an 
instant solution I am prepared to re-read some tortured syntax that I 
can understand) to make sites clearer. This may or may not come through 
in user testing - it is a good example of where the quality of the 
testing is more important than whether it is done by people with 
disabilities.

cheers

Chaals

On 15 Apr 2004, at 06:10, Ian Anderson wrote:

>> I'm not so sure. Unfortunately I have seen too many results by
>> reasonably substantial groups of testers that are as bad as anything 
>> an
>
> How do you define "good" as you have used the word here?
>
--
Charles McCathieNevile                          Fundación Sidar
charles@sidar.org                                http://www.sidar.org

Received on Wednesday, 14 April 2004 18:33:18 UTC