- From: Scott Luebking <phoenixl@sonic.net>
- Date: Sun, 12 Nov 2000 21:45:01 -0800
- To: w3c-wai-ig@w3.org
Hi, While I was going through the results of the online form errors tests, the problems that people had using the Palm Beach ballot began surfacing. The timing was quite surprising and brought out more strongly the concerns with regard to people making decisions and accurately specifying what they are. Some background for test: A couple of months ago someone from Henter-Joyce, a company which develops screen readers for blind people, contacted me about some of the experiments I've been doing concerning web technology and blind access. The discussion ended up being one about the limitations to the help that access technology can provide. Basically, while the technology can usually provide low-level access to information on the screen, the technology will probably be limited in helping blind people be efficient in accessing the information in a number of situations. For example, how does a person tell a screen reader to move to that object which is substantially different from the objects around it if the user is unfamiliar with the web page? This discussion occurred at the time I was developing an general information integrator in Java. While the work is unrelated to disability, some of the general approaches I've been using could be applied to the efficiency problem. One of the challenges that blind people encounter is fixing errors on a form. While a screen reader can help the blind user manipulate entries in a form, the screen reader often cannot identify very efficiently which form entries have errors. It can be analogous to a blind person being put into a room of furniture and being told to fix the broken table legs. While the individual most likely can perform the task, there is probably a difference in efficiency when compared to a sighted person performing the same task unless there is additional structure. Methodology: There are actually two parts to the methodology used on test web page. One was showing various ways that form errors could be presented. The idea was to get some feed-back on preferences in order to develop approaches to form error presentation which would be more universal. The qualitative approach used was basically to provide an open-ended comment field with little direction. The other part of the methodology was focused on trying to understand in what ways blind users are less efficient when fixing errors on forms. The web page used in the test had a mechanism in Javacript which recorded the time whenever a user changed a form element or selected an error navigation link. This mechanism created a log of the actions which could be examined at a later date for patterns. When the user selected the submit button, the activity log was returned along with the other information. One concern was that if the form was too long that people wouldn't finish it. The decision was to make it shorter by including only a few fields without errors. Results and Observations There were 65 forms were returned of which 21 were marked as being from blind people. There were 7 pieces of significant email from blind people. Three "stream of consciousness" phone calls with blind people while using the forms. Two informal visual observations of sighted people using the forms. In general, if the various sources of information were ranked according to usefulness, the order of most useful to less useful would be something like: 1. "stream or consciousness" phone calls and visual observation 2. email from people who used the test 3. comments in the test 4. logs of actions during the test Two issues about form errors were repeatedly mentioned. Several people complained about web pages presenting error messages about forms on a different page than the forms which frequently happens. They expressed positive opinions about the error messages being on the same page as the forms as used in the test. One of the discussions that came up was to what degree does having error messages on a page different from the form imopact users. This arrangement might have a greater impact on the efficiency of many blind form users than sighted form users. The difference lies in how information is transmitted to blind users as compared to sighted users. A sighted persion is given a much larger piece of information at a time than a blind person, i.e. an image versus a string of words. When the error messages are on a different page than the form, the sighted person can read an error message, go back to the previous page with the form, scan the page for the desired field with the error and fix the error. For many blind users, this arrangement can be much more cumbersome because they cannot scan over a page. They may have to start at the beginning of the page and have the page read to them till they get to the desired field. This process is frequently inefficient. Some blind people can compensate in certain ways. For example, some blind people can significantly increase the rate at which a web page is read which reduces the amount of time needed to get to the field with the error. Another way is to use any text search mechanism available, e.g. search for first name. Unfortunately, this mechanism is not always reliable. For example, suppose that there are two fields on a form for the first name each of the spouses in a marriage. Looking for the appropriate first name field can be a problem. Another complication would be if the web page had two forms, each of which had a first name field. Some blind people argue that many blind people can be taught to handle these types of problems. There are two questions. One is whether most blind people can be taught to compensate? The other is if they are taught to compensate, how much slower will they be than sighted people if there is any difference? The other issue that was brought up a number of times was that the arrangement of the error messages wasn't consistent. One purpose of the test was to try different error message arrangements. However, the impression I got was that while it was an annoyance for some sighted users, it seem to cause more confusion and reduced the efficiency for various blind users. Recovering from a confused state seemed to be more of a problem for various blind users than sighted users. The web page presented errors with two different approaches. One approach was to have links between the error messages so that a user can jump from error message to error message in a sequential order. The other approach was to have the error messages without the links. The sight users seemed divided about whether the linking was useful or not. Some sighted users recognized that the error messages were in yellow boxes so they could quickly scan for the yellow boxes. Other sighted users liked that the links between the error would scroll them to the next error. A sighted user commented that she scanned for the yellow error boxes because the errors were close together, but would find the links helpful if the errors were farther apart. Two sighted users who scanned commented that the linking was annoying. None of the blind users mentioned that they knew to scan for yellow boxes so it is not clear that they recognized that the error messages were in yellow. There was very favorable response from various blind users to using links to move from error to error though it was an idea that few had encountered before. Trying to find errors on a page can be not very efficient for many blind users. Unless the blind person is familiar with the web page, it can be risky to scan for the word "error" since error messages may not contain the word error. One blind user had a comment similar to one made by a sighted person that the density of the error messages affected the strategy she used. Another blind user commented how it was somewhat confusing because the test form first starts with not using links and then switches to using links. Another comment from various blind users was that the error messages should come before the field with the error rather than after like some fields on the test form. This arrangement increases the blind user's efficiency so that they don't have to back up looking for the field and then hear the error messages again when they move forward. A comment was made that error messages should be complete enough that the blind person doesn't have to go hunting around to develop a context for the error message, e.g. "Please make your selection." is not as clear as "Please indicate if the nuclear bomb should detonate today or not." There were also positive opinions on numbering the error messages. Many blind people can have problems knowing how many error messages there are on a page. They may have to read the whole form to be sure that they have found all the error messages. Recommendations: (Probably, more testing would be helpful to confirm these recommendatins.) 1. redisplay form with error messages instead of error messages on a separate page 2. put a summary of number of error messages as close to the beginning of a web page as possible. (this will help blind users more quickly identify what is happening with the page.) 3. put a link from the summary to the first error message area. (this can also be useful for sighted people who scan if they are unfamiliar with the web page.) 4. an error message should come before the field with errors 5. error messages should be clearly indicated by a color or an image to help people who are scanning. a combination of color and image is probably appropriate in case the user is color-blind 6. the error messages should be comprehensive enough so that a blind user doesn't have to look around the form for information to build a context for the error message 7. links should be provided after the error field to navigate to the previous or next errors. if the error is the last in the form, there should be a link to the submit buttons for the form. 8. error messages or links should be numbered e.g. (2/7) , to give a context when going through errors I found that implementing the error linking was actually fairly easily in both Perl And Java. Basically, I found that I could handle the problem by adding four tag types which were: <error-summary> <error-message-link-target> <form-error-check> <error-message-link> For example: <error-message-link-target> <input name=phone1 size=12> <br> Message <input name=phone1type type=checkbox value=message> Day <input name=phone1type type=checkbox value=day> Evening <input name=phone1type type=checkbox value=evening> TTD <input name=phone1type type=checkbox value=ttd> <form-error-check type=phone field=phone1 phonetyperequired> <error-message-link> In this example, the two tags <form-error-check> and <error-message-link> can be combined, e.g.: <error-message-link type=phone field=phone1 phonetyperequired> A container structure could have also been used. Just a matter of personal preference. Other observations: The testing also seemed to indicate that there are some significant differences between how many sighted and many blind people work with forms containing error messages (and probably web pages in general). The sighted users seemed to have: 1. greater ability to scan in a non-linear format 2. greater ability to jump over parts of the web page 3. greater ability to deduce context 4. less need for consistency 5. less preference for information density These observations are pretty informal and need to have more rigorous testing to see how strong the differences are and to what degree that blind users will be at disadvantages when compared to sighted users in terms of both efficiency and accuracy, The methodology used elicited some interesting qualitative information. If additional testing were done, it would probably be better to use two forms so that it would be less confusing. Also, it would be interesting to expand the forms to decrease the density of the error messages. This might create more interesting timing information in the action logs. Also, instead of one "open-ended" comment area, a series of questions about various aspects might have been better. There might be another problem which needs to be addressed in the forms testing. An impression I got is that there might be a difference in how sighted users and blind users perceive lengths of forms, e.g. what seems like an average length form for a number of sighted users might be seen as a long form for a number of blind users. It is not clear to me how often this might be true. Also, I'm not sure what would cause the diffetence in the perceptions. One conjecture is that it is harder for many blind users to use forms than sighted users and the amount of additional effort needed might make the forms appear longer. The action logs showed another interesting pattern. A number of the action logs were very short, i.e. very few actions before submitting. Only about 10% of the action logs from sighted users were short. About 50% of the action logs marked as being from blind users were short. There can be a technical reason for this that the default for user type was set to blind user. If a person submitted the form by key, the user type would default to blind. However, sighted people seem less likely to submit by key than blind users. If 50% of the blind users did generated short action logs, this could be indicative of problems that some blind users have with forms. One problem might be submitting a form by accident or ignorance. Another problem might be becoming frustrated more easily and taking a chance to see what happens. Given what happened on Palm Beach ballots and also the development of electronic signatures, there definitely needs to be more research and user testing about how well a wide range of blind people will be able to be efficient and accurate in handling forms. A concern I have is that both the technology world and the disabled community will put off this type of analysis until a situation develops which leads to some kind of lawsuit, e.g. a blind user specifying the wrong choice on a ballot or agreeing to a contract where some piece of vital information was overlooked. Scott
Received on Monday, 13 November 2000 00:45:35 UTC