W3C home > Mailing lists > Public > public-wsc-wg@w3.org > July 2007

Re: ACTION-232 OPEN Share results from his study once he has them

From: Mike Beltzner <beltzner@mozilla.com>
Date: Thu, 26 Jul 2007 13:26:27 -0400
Message-Id: <63924E60-3F8D-4C43-9056-A16209BA7BF6@mozilla.com>
Cc: Web Security Context WG <public-wsc-wg@w3.org>
To: Serge Egelman <egelman@cs.cmu.edu>

Serge, thanks for sharing these results. Were there any insights into  
the user's mental model when they hit a warning after expecting to  
arrive at a legitimate website? Did you get any feeling about whether  
or not the language used in the warnings had any effect in addition  
to the active vs. passive nature of the warnings?

Also, I think your link to the screenshot of Firefox2's anti-phishing  
warning is incorrect.

cheers,
mike

On 26-Jul-07, at 12:45 PM, Serge Egelman wrote:

>
> We conducted a study of active phishing indicators found in current  
> web
> browsers by simulating spear phishing attacks.  Active phishing
> indicators differ from passive indicators in that they interrupt the
> user's primary task, forcing a decision to be made.  Previous studies
> (no doubt you've read the Shared Bookmarks, right?) have shown that
> passive indicators often go unnoticed, and when they are noticed, are
> untrusted because users place more trust in the look and feel of the
> destination web page.  Both IE7 and Firefox 2 include active phishing
> warnings.
>
> Participants came to our lab under the guise of an online shopping
> study.  Purchases were made from Amazon and eBay using their own
> information.  Upon the completion of a purchase, participants were  
> sent
> phishing message from these sites, and were told to check their email
> accounts to make sure that their orders were confirmed.  Participants
> were then observed interacting with the phishing websites.   
> Participants
> were placed in one of four groups: 12 users of Firefox 2
> (http://switchersblog.com/files/firefox-phishing-protection.png), 10
> users of IE7 who were shown the passive warning
> (http://www.itwriting.com/images/localphishing.gif), 10 users of  
> IE7 who
> were shown the active phishing warning
> (http://www.billp.com/blog/images/ie7phishing.jpg), and a control  
> group
> (10 users) that was shown and phishing warnings.  The purpose of the
> control group was to determine whether participants would enter  
> personal
> information in the absence of a warning.
>
> Of the 42 participants, all but two individuals (one in the control
> group, one in the active IE7 group) clicked at least one of the  
> phishing
> URLs.  The 9 participants in the control group who clicked the URLs  
> all
> entered login information at the phishing sites.  9 participants in  
> the
> passive IE7 group entered login information (1 participant obeyed the
> warnings).  Participants ignored the passive warnings for two reasons:
> habituation with popup messages, and lack of choices in the dialog  
> (some
> participants read the warnings, but since there were no options, they
> were unsure of what to do, and thus dismissed the warnings and
> proceeded).  Additionally, some participants were so focused on the
> primary task (entering login information on the phishing websites)  
> that
> they did not notice the warnings appear in the first place.
>
> Among those shown the active warnings, all of the Firefox users obeyed
> the warnings.  In the active IE7 warning group, all but two  
> participants
> obeyed the warnings, however there was no statistically significant
> difference between these two groups.  Of the two who ignored the
> warnings, one blamed habituation, and the other was fooled by the
> message coinciding with the purchase.  This both shows that the IE7
> warning is designed too similar to other warnings in IE (e.g. the 404
> page), and that there will always be some users who fall for phishing
> attacks, regardless of the strength of the warnings.
>
> Overall, the active warnings were effective because they  
> interrupted the
> users' primary tasks ("attention switch") and they forced the users to
> make a choice in order to dismiss them ("attention maintenance").   
> These
> properties were lacking in the passive indicators.  Additionally, when
> visiting the eBay site, users were shown the EV certificate indicator
> (i.e. the green address bar) in IE7.  None of the 42 users noticed the
> green address bar, much less the absence of it when visiting the
> phishing sites.  Thus, it is unreasonable to expect users to be warned
> by the absence of an indicator.
>
> We also found that prior experiences with phishing had zero  
> correlation
> with falling for a phishing attack in our study.  One third of the
> participants claimed to have either fallen for a phishing attack, had
> credentials stolen, or been the victim of credit fraud in the past.
> These individuals were equally as likely to both click on the URLs and
> ignore the warnings as other participants.  Additionally, participants
> who could define the term "phishing" were not anymore likely to  
> obey (or
> ignore) the warnings than participants who could not.  Finally, when
> asked how they believed the phishing messages got to them,  
> participants
> could not answer.  They understood the websites were fraudulent,  
> however
> they still trusted the email messages.  This shows that there is a  
> huge
> disconnect with users' mental models of phishing.
>
> Overall we concluded that warnings within the phishing context need to
> interrupt the user's primary task to be effective.  These warnings  
> must
> present clear recommendations on how to proceed.  To prevent
> habituation, these warnings should be designed differently than  
> dialogs
> and need to be presented rarely (i.e. only when there's a high
> probability of immediate danger).  Finally, warnings about high risks
> need to fail safely, for when users do become habituated.  One
> participant in this study who was exposed to the active IE7 warning  
> did
> not read it (or the options it presented), and thus clicked the red  
> 'X'
> in the corner to dismiss it (thus closing the browser window).  She  
> went
> back to the original email, clicked the link again, and again  
> closed the
> window.  She repeated this process five times before finally giving  
> up,
> and was thus prevented from giving away information to the phishing
> website despite the fact that she never read any part of the warning.
>
> If you have any questions, feel free to ask.  I'm still working on the
> paper.
>
>
> serge
>
>
> -- 
> /*
> Serge Egelman
>
> PhD Candidate
> Vice President for External Affairs, Graduate Student Assembly
> Carnegie Mellon University
>
> Legislative Concerns Chair
> National Association of Graduate-Professional Students
> */
>
Received on Thursday, 26 July 2007 17:26:52 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 5 February 2008 03:52:50 GMT