Comments from Fraunhofer FIT to the document "Evaluating Web Accessibility with Users" (2005-09-02)

This document contains some initial comments to the existing draft of Evaluating Web Accessibility with Users. We are aware that this is an early draft, but we found some key points that should be handled with care (Henrike Gappa, Gabriele Nordbrock).

In regard to Introduction and Types of User Involvement

"Encouraging ..."

In regard to section The User Aspect

"More than screen reader users ..."

In regard to section Understanding [Finding/Results]

"One user not representative of all: ..."

It needs to be assured that test results are of substantial use, so generally speaking gaining reliable test results should be the goal. However, depending on the aim of the study and the method chosen, this might differ. The characteristics of the test participants and the size of the test sample is tobe determined according to the premises of the user study (test sample size is an issue yet unsolved: for usability studies a test sample size of 10-12 seems to be a common understanding, in the field of psychology, 30 test participants is considered the minimum. In case the "5 users are enough"-paradigma of Nielsen is used, the drawback must be considered.

Distinction between usability and accessibility issues: the distinction between usability and accessibility issues is from our point of view not feasible, because:

Clearly indicate what the report asserts: