W3C home > Mailing lists > Public > public-wsc-wg@w3.org > July 2007

Re: ACTION-232 OPEN Share results from his study once he has them

From: Serge Egelman <egelman@cs.cmu.edu>
Date: Thu, 26 Jul 2007 13:52:42 -0400
Message-ID: <46A8DF6A.6030205@cs.cmu.edu>
To: Mike Beltzner <beltzner@mozilla.com>
CC: Web Security Context WG <public-wsc-wg@w3.org>

What's wrong with the screenshot?  It shows up correctly for me....


Mike Beltzner wrote:
> Serge, thanks for sharing these results. Were there any insights into
> the user's mental model when they hit a warning after expecting to
> arrive at a legitimate website? Did you get any feeling about whether or
> not the language used in the warnings had any effect in addition to the
> active vs. passive nature of the warnings?
> Also, I think your link to the screenshot of Firefox2's anti-phishing
> warning is incorrect.
> cheers,
> mike
> On 26-Jul-07, at 12:45 PM, Serge Egelman wrote:
>> We conducted a study of active phishing indicators found in current web
>> browsers by simulating spear phishing attacks.  Active phishing
>> indicators differ from passive indicators in that they interrupt the
>> user's primary task, forcing a decision to be made.  Previous studies
>> (no doubt you've read the Shared Bookmarks, right?) have shown that
>> passive indicators often go unnoticed, and when they are noticed, are
>> untrusted because users place more trust in the look and feel of the
>> destination web page.  Both IE7 and Firefox 2 include active phishing
>> warnings.
>> Participants came to our lab under the guise of an online shopping
>> study.  Purchases were made from Amazon and eBay using their own
>> information.  Upon the completion of a purchase, participants were sent
>> phishing message from these sites, and were told to check their email
>> accounts to make sure that their orders were confirmed.  Participants
>> were then observed interacting with the phishing websites.  Participants
>> were placed in one of four groups: 12 users of Firefox 2
>> (http://switchersblog.com/files/firefox-phishing-protection.png), 10
>> users of IE7 who were shown the passive warning
>> (http://www.itwriting.com/images/localphishing.gif), 10 users of IE7 who
>> were shown the active phishing warning
>> (http://www.billp.com/blog/images/ie7phishing.jpg), and a control group
>> (10 users) that was shown and phishing warnings.  The purpose of the
>> control group was to determine whether participants would enter personal
>> information in the absence of a warning.
>> Of the 42 participants, all but two individuals (one in the control
>> group, one in the active IE7 group) clicked at least one of the phishing
>> URLs.  The 9 participants in the control group who clicked the URLs all
>> entered login information at the phishing sites.  9 participants in the
>> passive IE7 group entered login information (1 participant obeyed the
>> warnings).  Participants ignored the passive warnings for two reasons:
>> habituation with popup messages, and lack of choices in the dialog (some
>> participants read the warnings, but since there were no options, they
>> were unsure of what to do, and thus dismissed the warnings and
>> proceeded).  Additionally, some participants were so focused on the
>> primary task (entering login information on the phishing websites) that
>> they did not notice the warnings appear in the first place.
>> Among those shown the active warnings, all of the Firefox users obeyed
>> the warnings.  In the active IE7 warning group, all but two participants
>> obeyed the warnings, however there was no statistically significant
>> difference between these two groups.  Of the two who ignored the
>> warnings, one blamed habituation, and the other was fooled by the
>> message coinciding with the purchase.  This both shows that the IE7
>> warning is designed too similar to other warnings in IE (e.g. the 404
>> page), and that there will always be some users who fall for phishing
>> attacks, regardless of the strength of the warnings.
>> Overall, the active warnings were effective because they interrupted the
>> users' primary tasks ("attention switch") and they forced the users to
>> make a choice in order to dismiss them ("attention maintenance").  These
>> properties were lacking in the passive indicators.  Additionally, when
>> visiting the eBay site, users were shown the EV certificate indicator
>> (i.e. the green address bar) in IE7.  None of the 42 users noticed the
>> green address bar, much less the absence of it when visiting the
>> phishing sites.  Thus, it is unreasonable to expect users to be warned
>> by the absence of an indicator.
>> We also found that prior experiences with phishing had zero correlation
>> with falling for a phishing attack in our study.  One third of the
>> participants claimed to have either fallen for a phishing attack, had
>> credentials stolen, or been the victim of credit fraud in the past.
>> These individuals were equally as likely to both click on the URLs and
>> ignore the warnings as other participants.  Additionally, participants
>> who could define the term "phishing" were not anymore likely to obey (or
>> ignore) the warnings than participants who could not.  Finally, when
>> asked how they believed the phishing messages got to them, participants
>> could not answer.  They understood the websites were fraudulent, however
>> they still trusted the email messages.  This shows that there is a huge
>> disconnect with users' mental models of phishing.
>> Overall we concluded that warnings within the phishing context need to
>> interrupt the user's primary task to be effective.  These warnings must
>> present clear recommendations on how to proceed.  To prevent
>> habituation, these warnings should be designed differently than dialogs
>> and need to be presented rarely (i.e. only when there's a high
>> probability of immediate danger).  Finally, warnings about high risks
>> need to fail safely, for when users do become habituated.  One
>> participant in this study who was exposed to the active IE7 warning did
>> not read it (or the options it presented), and thus clicked the red 'X'
>> in the corner to dismiss it (thus closing the browser window).  She went
>> back to the original email, clicked the link again, and again closed the
>> window.  She repeated this process five times before finally giving up,
>> and was thus prevented from giving away information to the phishing
>> website despite the fact that she never read any part of the warning.
>> If you have any questions, feel free to ask.  I'm still working on the
>> paper.
>> serge
>> --/*
>> Serge Egelman
>> PhD Candidate
>> Vice President for External Affairs, Graduate Student Assembly
>> Carnegie Mellon University
>> Legislative Concerns Chair
>> National Association of Graduate-Professional Students
>> */

Serge Egelman

PhD Candidate
Vice President for External Affairs, Graduate Student Assembly
Carnegie Mellon University

Legislative Concerns Chair
National Association of Graduate-Professional Students
Received on Thursday, 26 July 2007 17:53:05 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:17 UTC