Fwd: [Scott Luebking] Some results from testing forms errors

[An interesting experimental study by Scott Luebking - Al]

[headers at foot]
>
>Hi,
>
>While I was going through the results of the online form errors tests,
>the problems that people had using the Palm Beach ballot began surfacing.
>The timing was quite surprising and brought out more strongly the
>concerns with regard to people making decisions and accurately specifying
>what they are.
>
>
>Some background for test:
>
>A couple of months ago someone from Henter-Joyce, a company which
>develops screen readers for blind people, contacted me about some of the
>experiments I've been doing concerning web technology and blind access.
>The discussion ended up being one about the limitations to the help that
>access technology can provide.  Basically, while the technology can
>usually provide low-level access to information on the screen, the
>technology will probably be limited in helping blind people be efficient
>in accessing the information in a number of situations.  For example,
>how does a person tell a screen reader to move to that object which is
>substantially different from the objects around it if the user is
>unfamiliar with the web page?
>
>This discussion occurred at the time I was developing an general
>information integrator in Java.  While the work is unrelated to
>disability, some of the general approaches I've been using could be
>applied to the efficiency problem.
>
>One of the challenges that blind people encounter is fixing errors on a
>form.  While a screen reader can help the blind user manipulate entries
>in a form, the screen reader often cannot identify very efficiently
>which form entries have errors.  It can be analogous to a blind person
>being put into a room of furniture and being told to fix the broken
>table legs.  While the individual most likely can perform the task,
>there is probably a difference in efficiency when compared to a sighted
>person performing the same task unless there is additional structure.
>
>
>Methodology:
>
>There are actually two parts to the methodology used on test web page.
>One was showing various ways that form errors could be presented.  The
>idea was to get some feed-back on preferences in order to develop
>approaches to form error presentation which would be more universal.
>The qualitative approach used was basically to provide an open-ended
>comment field with little direction.
>
>The other part of the methodology was focused on trying to understand in
>what ways blind users are less efficient when fixing errors on forms.
>The web page used in the test had a mechanism in Javacript which recorded
>the time whenever a user changed a form element or selected an error
>navigation link.  This mechanism created a log of the actions which
>could be examined at a later date for patterns.  When the user selected
>the submit button, the activity log was returned along with the other
>information.
>
>One concern was that if the form was too long that people wouldn't finish it.
>The decision was to make it shorter by including only a few fields without
>errors.
>
>
>Results and Observations
>
>There were 65 forms were returned of which 21 were marked as being from
>blind people.  There were 7 pieces of significant email from blind
>people.  Three "stream of consciousness" phone calls with blind people
>while using the forms.  Two informal visual observations of sighted people
>using the forms.  In general, if the various sources of information
>were ranked according to usefulness, the order of most useful to less
>useful would be something like:
>
>     1.  "stream or consciousness" phone calls and visual observation
>
>     2.  email from people who used the test
>
>     3.  comments in the test
>
>     4.  logs of actions during the test
>
>
>Two issues about form errors were repeatedly mentioned.  Several people
>complained about web pages presenting error messages about forms on a
>different page than the forms which frequently happens.  They expressed
>positive opinions about the error messages being on the same page as the
>forms as used in the test.
>
>One of the discussions that came up was to what degree does having error
>messages on a page different from the form imopact users.  This
>arrangement might have a greater impact on the efficiency of many blind
>form users than sighted form users.  The difference lies in how
>information is transmitted to blind users as compared to sighted users.
>A sighted persion is given a much larger piece of information at a time
>than a blind person, i.e.  an image versus a string of words.  When the
>error messages are on a different page than the form, the sighted person
>can read an error message, go back to the previous page with the form,
>scan the page for the desired field with the error and fix the error.
>For many blind users, this arrangement can be much more cumbersome
>because they cannot scan over a page.  They may have to start at the
>beginning of the page and have the page read to them till they get to
>the desired field.  This process is frequently inefficient.
>
>Some blind people can compensate in certain ways.  For example, some
>blind people can significantly increase the rate at which a web page is
>read which reduces the amount of time needed to get to the field with
>the error.  Another way is to use any text search mechanism available,
>e.g. search for first name.  Unfortunately, this mechanism is not always
>reliable.  For example, suppose that there are two fields on a form for
>the first name each of the spouses in a marriage.  Looking for the
>appropriate first name field can be a problem.  Another complication
>would be if the web page had two forms, each of which had a first name
>field.
>
>Some blind people argue that many blind people can be taught to handle these
>types of problems.  There are two questions.  One is whether most blind
>people can be taught to compensate?  The other is if they are taught to
>compensate, how much slower will they be than sighted people if there is
>any difference?
>
>
>
>The other issue that was brought up a number of times was that the
>arrangement of the error messages wasn't consistent.  One purpose of the
>test was to try different error message arrangements.  However, the
>impression I got was that while it was an annoyance for some sighted
>users, it seem to cause more confusion and reduced the efficiency for
>various blind users.  Recovering from a confused state seemed to be more of
>a problem for various blind users than sighted users.
>
>
>The web page presented errors with two different approaches.  One
>approach was to have links between the error messages so that a user can
>jump from error message to error message in a sequential order.  The
>other approach was to have the error messages without the links.
>
>The sight users seemed divided about whether the linking was useful or
>not.  Some sighted users recognized that the error messages were in
>yellow boxes so they could quickly scan for the yellow boxes.  Other
>sighted users liked that the links between the error would scroll them
>to the next error.  A sighted user commented that she scanned for the
>yellow error boxes because the errors were close together, but would
>find the links helpful if the errors were farther apart.  Two sighted
>users who scanned commented that the linking was annoying.
>
>
>None of the blind users mentioned that they knew to scan for yellow
>boxes so it is not clear that they recognized that the error messages
>were in yellow.  There was very favorable response from various blind
users to
>using links to move from error to error though it was an idea that few
>had encountered before.  Trying to find errors on a page can be not very
>efficient for many blind users.  Unless the blind person is familiar with
>the web page, it can be risky to scan for the word "error" since error
>messages may not contain the word error.  One blind user had a comment
>similar to one made by a sighted person that the density of the error
>messages affected the strategy she used.  Another blind user commented how it
>was somewhat confusing because the test form first starts with not using
>links and then switches to using links.
>
>Another comment from various blind users was that the error messages
>should come before the field with the error rather than after like some
>fields on the test form. This arrangement increases the blind user's
efficiency
>so that they don't have to back up looking for the field and then hear the
>error messages again when they move forward.
>
>A comment was made that error messages should be complete enough that
>the blind person doesn't have to go hunting around to develop a context
>for the error message, e.g. "Please make your selection." is not as
>clear as "Please indicate if the nuclear bomb should detonate today or
>not."
>
>There were also positive opinions on numbering the error messages.  Many
>blind people can have problems knowing how many error messages there are
>on a page.  They may have to read the whole form to be sure that they
>have found all the error messages.
>
>
>
>Recommendations:
>
>(Probably, more testing would be helpful to confirm these recommendatins.)
>
>     1.  redisplay form with error messages instead of error messages on a
>         separate page
>
>     2.  put a summary of number of error messages as close to the beginning
of
>         a web page as possible.  (this will help blind users
> more quickly identify what is happening with the page.)
>
>     3.  put a link from the summary to the first error message area.
>         (this can also be useful for sighted people who scan if they
> are unfamiliar with the web page.)
>
>     4.  an error message should come before the field with errors
>
>     5.  error messages should be clearly indicated by a color or an
>         image to help people who are scanning.  a combination of color
> and image is probably appropriate in case the user is color-blind
>
>     6.  the error messages should be comprehensive enough so that a
>         blind user doesn't have to look around the form for information
> to build a context for the error message
>
>     7.  links should be provided after the error field to navigate
>         to the previous or next errors. if the error is the last in the
> form, there should be a link to the submit buttons for the form.
>
>     8.  error messages or links should be numbered e.g. (2/7) , to give a
>         context when going through errors
>
>
>I found that implementing the error linking was actually fairly easily in
>both Perl And Java.  Basically, I found that I could handle the problem
>by adding four tag types which were:
>
>     <error-summary>
>     <error-message-link-target>
>     <form-error-check>
>     <error-message-link>
>
>For example:
>
>     <error-message-link-target>
>     <input name=phone1 size=12>
>     <br>
>     Message <input name=phone1type type=checkbox value=message>
>     Day <input name=phone1type type=checkbox value=day>
>     Evening <input name=phone1type type=checkbox value=evening>
>     TTD <input name=phone1type type=checkbox value=ttd>
>     <form-error-check type=phone field=phone1 phonetyperequired>
>     <error-message-link>
>
>In this example, the two tags <form-error-check> and <error-message-link>
>can be combined, e.g.:
>
>     <error-message-link type=phone field=phone1 phonetyperequired>
>
>A container structure could have also been used.  Just a matter of personal
>preference.
>
>
>
>
>Other observations:
>
>The testing also seemed to indicate that there are some significant
>differences between how many sighted and many blind people work with forms
>containing error messages (and probably web pages in general).  The sighted
>users seemed to have:
>
>     1.  greater ability to scan in a non-linear format
>
>     2.  greater ability to jump over parts of the web page
>
>     3.  greater ability to deduce context
>
>     4.  less need for consistency
>
>     5.  less preference for information density
>
>These observations are pretty informal and need to have more rigorous testing
>to see how strong the differences are and to what degree that blind users
>will be at disadvantages when compared to sighted users in terms of both
>efficiency and accuracy,
>
>
>The methodology used elicited some interesting qualitative information.
>If additional testing were done, it would probably be better to use
>two forms so that it would be less confusing.  Also, it would be interesting
>to expand the forms to decrease the density of the error messages.
>This might create more interesting timing information in the action logs.
>Also, instead of one "open-ended" comment area, a series of questions
>about various aspects might have been better.
>
>
>There might be another problem which needs to be addressed in the forms
>testing.  An impression I got is that there might be a difference
>in how sighted users and blind users perceive lengths of forms, e.g.
>what seems like an average length form for a number of sighted users
>might be seen as a long form for a number of blind users.  It is
>not clear to me how often this might be true.  Also, I'm not sure
>what would cause the diffetence in the perceptions.  One conjecture is that
>it is harder for many blind users to use forms than sighted users and
>the amount of additional effort needed might make the forms appear longer.
>
>The action logs showed another interesting pattern.  A number of the action
>logs were very short, i.e. very few actions before submitting.  Only about
>10% of the action logs from sighted users were short.  About 50% of the
action
>logs marked as being from blind users were short.  There can be a technical
>reason for this that the default for user type was set to blind user.
>If a person submitted the form by key, the user type would default to
>blind.  However, sighted people seem less likely to submit by key than
>blind users.  If 50% of the blind users did generated short action logs,
>this could be indicative of problems that some blind users have with forms.
>One problem might be submitting a form by accident or ignorance.  Another
>problem might be becoming frustrated more easily and taking a chance to
>see what happens.
>
>Given what happened on Palm Beach ballots and also the development of
>electronic signatures, there definitely needs to be more research and user
>testing about how well a wide range of blind people will be able to be
>efficient and accurate in handling forms.  A concern I have is that both
>the technology world and the disabled community will put off this type of
>analysis until a situation develops which leads to some kind of lawsuit,
>e.g. a blind user specifying the wrong choice on a ballot or agreeing
>to a contract where some piece of vital information was overlooked.
>
>
>Scott
>
>
>
>Community email addresses:
>   Post message: ba-univ-tech-access@onelist.com
>   Subscribe:    ba-univ-tech-access-subscribe@onelist.com
>   Unsubscribe:  ba-univ-tech-access-unsubscribe@onelist.com
>   List owner:   ba-univ-tech-access-owner@onelist.com
>
>Shortcut URL to this page:
>  
<http://www.onelist.com/community/ba-univ-tech-access>http://www.onelist.com
/community/ba-univ-tech-access
>
>
>-------------------------- eGroups Sponsor -------------------------~-~>
>eGroups eLerts
>It's Easy. It's Fun. Best of All, it's Free!
><http://click.egroups.com/1/9698/1/_/68434/_/974100297/>http://click.egrou
ps.com/1/9698/1/_/68434/_/974100297/
>---------------------------------------------------------------------_->
>
>To Post a message, send it to:   webwatch@eGroups.com
>To Unsubscribe, send a blank message to: webwatch-unsubscribe@eGroups.com
> 
>X-eGroups-Return:
sentto-1014776-10-974100297-asgilman=iamdigex.net@returns.onelist.com
>X-Sender: kford@teleport.com
>X-Apparently-To: webwatch@egroups.com
>X-Sender: kford@mail.teleport.com
>X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 
>To: webwatch@egroups.com
>X-eGroups-From: Scott Luebking <phoenixl@sonic.net>(by way of Kelly Ford
<kford@teleport.com>)
>From: Scott Luebking <kford@teleport.com>(by way of Kelly Ford
<kford@teleport.com>)
>Mailing-List: list webwatch@egroups.com; contact webwatch-owner@egroups.com
>Delivered-To: mailing list webwatch@egroups.com
>List-Unsubscribe:
<<mailto:webwatch-unsubscribe@egroups.com>mailto:webwatch-unsubscribe@egroup
s.com>
>Date: Sun, 12 Nov 2000 23:47:09 -0800
>Reply-To: webwatch@egroups.com
>Subject: [webwatch] Some results from testing forms errors

Received on Monday, 13 November 2000 08:02:39 UTC