- From: Al Gilman <asgilman@iamdigex.net>
- Date: Mon, 13 Nov 2000 08:33:29 -0500
- To: w3c-wai-er-ig@w3.org
[An interesting experimental study by Scott Luebking - Al] [headers at foot] > >Hi, > >While I was going through the results of the online form errors tests, >the problems that people had using the Palm Beach ballot began surfacing. >The timing was quite surprising and brought out more strongly the >concerns with regard to people making decisions and accurately specifying >what they are. > > >Some background for test: > >A couple of months ago someone from Henter-Joyce, a company which >develops screen readers for blind people, contacted me about some of the >experiments I've been doing concerning web technology and blind access. >The discussion ended up being one about the limitations to the help that >access technology can provide. Basically, while the technology can >usually provide low-level access to information on the screen, the >technology will probably be limited in helping blind people be efficient >in accessing the information in a number of situations. For example, >how does a person tell a screen reader to move to that object which is >substantially different from the objects around it if the user is >unfamiliar with the web page? > >This discussion occurred at the time I was developing an general >information integrator in Java. While the work is unrelated to >disability, some of the general approaches I've been using could be >applied to the efficiency problem. > >One of the challenges that blind people encounter is fixing errors on a >form. While a screen reader can help the blind user manipulate entries >in a form, the screen reader often cannot identify very efficiently >which form entries have errors. It can be analogous to a blind person >being put into a room of furniture and being told to fix the broken >table legs. While the individual most likely can perform the task, >there is probably a difference in efficiency when compared to a sighted >person performing the same task unless there is additional structure. > > >Methodology: > >There are actually two parts to the methodology used on test web page. >One was showing various ways that form errors could be presented. The >idea was to get some feed-back on preferences in order to develop >approaches to form error presentation which would be more universal. >The qualitative approach used was basically to provide an open-ended >comment field with little direction. > >The other part of the methodology was focused on trying to understand in >what ways blind users are less efficient when fixing errors on forms. >The web page used in the test had a mechanism in Javacript which recorded >the time whenever a user changed a form element or selected an error >navigation link. This mechanism created a log of the actions which >could be examined at a later date for patterns. When the user selected >the submit button, the activity log was returned along with the other >information. > >One concern was that if the form was too long that people wouldn't finish it. >The decision was to make it shorter by including only a few fields without >errors. > > >Results and Observations > >There were 65 forms were returned of which 21 were marked as being from >blind people. There were 7 pieces of significant email from blind >people. Three "stream of consciousness" phone calls with blind people >while using the forms. Two informal visual observations of sighted people >using the forms. In general, if the various sources of information >were ranked according to usefulness, the order of most useful to less >useful would be something like: > > 1. "stream or consciousness" phone calls and visual observation > > 2. email from people who used the test > > 3. comments in the test > > 4. logs of actions during the test > > >Two issues about form errors were repeatedly mentioned. Several people >complained about web pages presenting error messages about forms on a >different page than the forms which frequently happens. They expressed >positive opinions about the error messages being on the same page as the >forms as used in the test. > >One of the discussions that came up was to what degree does having error >messages on a page different from the form imopact users. This >arrangement might have a greater impact on the efficiency of many blind >form users than sighted form users. The difference lies in how >information is transmitted to blind users as compared to sighted users. >A sighted persion is given a much larger piece of information at a time >than a blind person, i.e. an image versus a string of words. When the >error messages are on a different page than the form, the sighted person >can read an error message, go back to the previous page with the form, >scan the page for the desired field with the error and fix the error. >For many blind users, this arrangement can be much more cumbersome >because they cannot scan over a page. They may have to start at the >beginning of the page and have the page read to them till they get to >the desired field. This process is frequently inefficient. > >Some blind people can compensate in certain ways. For example, some >blind people can significantly increase the rate at which a web page is >read which reduces the amount of time needed to get to the field with >the error. Another way is to use any text search mechanism available, >e.g. search for first name. Unfortunately, this mechanism is not always >reliable. For example, suppose that there are two fields on a form for >the first name each of the spouses in a marriage. Looking for the >appropriate first name field can be a problem. Another complication >would be if the web page had two forms, each of which had a first name >field. > >Some blind people argue that many blind people can be taught to handle these >types of problems. There are two questions. One is whether most blind >people can be taught to compensate? The other is if they are taught to >compensate, how much slower will they be than sighted people if there is >any difference? > > > >The other issue that was brought up a number of times was that the >arrangement of the error messages wasn't consistent. One purpose of the >test was to try different error message arrangements. However, the >impression I got was that while it was an annoyance for some sighted >users, it seem to cause more confusion and reduced the efficiency for >various blind users. Recovering from a confused state seemed to be more of >a problem for various blind users than sighted users. > > >The web page presented errors with two different approaches. One >approach was to have links between the error messages so that a user can >jump from error message to error message in a sequential order. The >other approach was to have the error messages without the links. > >The sight users seemed divided about whether the linking was useful or >not. Some sighted users recognized that the error messages were in >yellow boxes so they could quickly scan for the yellow boxes. Other >sighted users liked that the links between the error would scroll them >to the next error. A sighted user commented that she scanned for the >yellow error boxes because the errors were close together, but would >find the links helpful if the errors were farther apart. Two sighted >users who scanned commented that the linking was annoying. > > >None of the blind users mentioned that they knew to scan for yellow >boxes so it is not clear that they recognized that the error messages >were in yellow. There was very favorable response from various blind users to >using links to move from error to error though it was an idea that few >had encountered before. Trying to find errors on a page can be not very >efficient for many blind users. Unless the blind person is familiar with >the web page, it can be risky to scan for the word "error" since error >messages may not contain the word error. One blind user had a comment >similar to one made by a sighted person that the density of the error >messages affected the strategy she used. Another blind user commented how it >was somewhat confusing because the test form first starts with not using >links and then switches to using links. > >Another comment from various blind users was that the error messages >should come before the field with the error rather than after like some >fields on the test form. This arrangement increases the blind user's efficiency >so that they don't have to back up looking for the field and then hear the >error messages again when they move forward. > >A comment was made that error messages should be complete enough that >the blind person doesn't have to go hunting around to develop a context >for the error message, e.g. "Please make your selection." is not as >clear as "Please indicate if the nuclear bomb should detonate today or >not." > >There were also positive opinions on numbering the error messages. Many >blind people can have problems knowing how many error messages there are >on a page. They may have to read the whole form to be sure that they >have found all the error messages. > > > >Recommendations: > >(Probably, more testing would be helpful to confirm these recommendatins.) > > 1. redisplay form with error messages instead of error messages on a > separate page > > 2. put a summary of number of error messages as close to the beginning of > a web page as possible. (this will help blind users > more quickly identify what is happening with the page.) > > 3. put a link from the summary to the first error message area. > (this can also be useful for sighted people who scan if they > are unfamiliar with the web page.) > > 4. an error message should come before the field with errors > > 5. error messages should be clearly indicated by a color or an > image to help people who are scanning. a combination of color > and image is probably appropriate in case the user is color-blind > > 6. the error messages should be comprehensive enough so that a > blind user doesn't have to look around the form for information > to build a context for the error message > > 7. links should be provided after the error field to navigate > to the previous or next errors. if the error is the last in the > form, there should be a link to the submit buttons for the form. > > 8. error messages or links should be numbered e.g. (2/7) , to give a > context when going through errors > > >I found that implementing the error linking was actually fairly easily in >both Perl And Java. Basically, I found that I could handle the problem >by adding four tag types which were: > > <error-summary> > <error-message-link-target> > <form-error-check> > <error-message-link> > >For example: > > <error-message-link-target> > <input name=phone1 size=12> > <br> > Message <input name=phone1type type=checkbox value=message> > Day <input name=phone1type type=checkbox value=day> > Evening <input name=phone1type type=checkbox value=evening> > TTD <input name=phone1type type=checkbox value=ttd> > <form-error-check type=phone field=phone1 phonetyperequired> > <error-message-link> > >In this example, the two tags <form-error-check> and <error-message-link> >can be combined, e.g.: > > <error-message-link type=phone field=phone1 phonetyperequired> > >A container structure could have also been used. Just a matter of personal >preference. > > > > >Other observations: > >The testing also seemed to indicate that there are some significant >differences between how many sighted and many blind people work with forms >containing error messages (and probably web pages in general). The sighted >users seemed to have: > > 1. greater ability to scan in a non-linear format > > 2. greater ability to jump over parts of the web page > > 3. greater ability to deduce context > > 4. less need for consistency > > 5. less preference for information density > >These observations are pretty informal and need to have more rigorous testing >to see how strong the differences are and to what degree that blind users >will be at disadvantages when compared to sighted users in terms of both >efficiency and accuracy, > > >The methodology used elicited some interesting qualitative information. >If additional testing were done, it would probably be better to use >two forms so that it would be less confusing. Also, it would be interesting >to expand the forms to decrease the density of the error messages. >This might create more interesting timing information in the action logs. >Also, instead of one "open-ended" comment area, a series of questions >about various aspects might have been better. > > >There might be another problem which needs to be addressed in the forms >testing. An impression I got is that there might be a difference >in how sighted users and blind users perceive lengths of forms, e.g. >what seems like an average length form for a number of sighted users >might be seen as a long form for a number of blind users. It is >not clear to me how often this might be true. Also, I'm not sure >what would cause the diffetence in the perceptions. One conjecture is that >it is harder for many blind users to use forms than sighted users and >the amount of additional effort needed might make the forms appear longer. > >The action logs showed another interesting pattern. A number of the action >logs were very short, i.e. very few actions before submitting. Only about >10% of the action logs from sighted users were short. About 50% of the action >logs marked as being from blind users were short. There can be a technical >reason for this that the default for user type was set to blind user. >If a person submitted the form by key, the user type would default to >blind. However, sighted people seem less likely to submit by key than >blind users. If 50% of the blind users did generated short action logs, >this could be indicative of problems that some blind users have with forms. >One problem might be submitting a form by accident or ignorance. Another >problem might be becoming frustrated more easily and taking a chance to >see what happens. > >Given what happened on Palm Beach ballots and also the development of >electronic signatures, there definitely needs to be more research and user >testing about how well a wide range of blind people will be able to be >efficient and accurate in handling forms. A concern I have is that both >the technology world and the disabled community will put off this type of >analysis until a situation develops which leads to some kind of lawsuit, >e.g. a blind user specifying the wrong choice on a ballot or agreeing >to a contract where some piece of vital information was overlooked. > > >Scott > > > >Community email addresses: > Post message: ba-univ-tech-access@onelist.com > Subscribe: ba-univ-tech-access-subscribe@onelist.com > Unsubscribe: ba-univ-tech-access-unsubscribe@onelist.com > List owner: ba-univ-tech-access-owner@onelist.com > >Shortcut URL to this page: > <http://www.onelist.com/community/ba-univ-tech-access>http://www.onelist.com /community/ba-univ-tech-access > > >-------------------------- eGroups Sponsor -------------------------~-~> >eGroups eLerts >It's Easy. It's Fun. Best of All, it's Free! ><http://click.egroups.com/1/9698/1/_/68434/_/974100297/>http://click.egrou ps.com/1/9698/1/_/68434/_/974100297/ >---------------------------------------------------------------------_-> > >To Post a message, send it to: webwatch@eGroups.com >To Unsubscribe, send a blank message to: webwatch-unsubscribe@eGroups.com > >X-eGroups-Return: sentto-1014776-10-974100297-asgilman=iamdigex.net@returns.onelist.com >X-Sender: kford@teleport.com >X-Apparently-To: webwatch@egroups.com >X-Sender: kford@mail.teleport.com >X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 >To: webwatch@egroups.com >X-eGroups-From: Scott Luebking <phoenixl@sonic.net>(by way of Kelly Ford <kford@teleport.com>) >From: Scott Luebking <kford@teleport.com>(by way of Kelly Ford <kford@teleport.com>) >Mailing-List: list webwatch@egroups.com; contact webwatch-owner@egroups.com >Delivered-To: mailing list webwatch@egroups.com >List-Unsubscribe: <<mailto:webwatch-unsubscribe@egroups.com>mailto:webwatch-unsubscribe@egroup s.com> >Date: Sun, 12 Nov 2000 23:47:09 -0800 >Reply-To: webwatch@egroups.com >Subject: [webwatch] Some results from testing forms errors
Received on Monday, 13 November 2000 08:02:39 UTC