Re: ACTION-232 OPEN Share results from his study once he has them

The phishing messages said something along the lines of the order would
be canceled unless they clicked the link to "confirm" it.  One of the
interesting tells about the mental model was that after obeying the
warnings and closing the window, most users said something along the
lines of "I guess this means the order will be canceled."  Thus
indicating that they trusted the phishing email, but also trusted the
warning message (and believed that the website was fraudulent).  Very
few of them could explain how they arrived at the fraudulent website.

Most of them did not read the text of the warning.  They read the
heading and the options.  So the detailed description is less important,
and really only relevant to a small number of users who a) have never
seen the warning before and b) are genuinely curious in learning more.
Both warnings have big text at the top which say something along the
lines of "Suspected Phishing Website."  This text was enough of a
deterrent for most users.  They read this, then skip to the options
because they understand there's a danger and want guidance on what to do
about it, and are less concerned with how it originated.

With the passive warning, it also said something along these lines at
the top.  The detailed description was also fairly similar to the
descriptions in the active warnings.  The main difference is that it
offered no guidance on what to do about it.  They understood there was
some sort of vague threat, but without guidance, they just dismissed it.
 The choices presented may also help to qualify the seriousness of the
danger (e.g. "if it's suggesting I close the window, this is probably
pretty serious").

serge

Mike Beltzner wrote:
> 
> Serge, thanks for sharing these results. Were there any insights into
> the user's mental model when they hit a warning after expecting to
> arrive at a legitimate website? Did you get any feeling about whether or
> not the language used in the warnings had any effect in addition to the
> active vs. passive nature of the warnings?
> 
> Also, I think your link to the screenshot of Firefox2's anti-phishing
> warning is incorrect.
> 
> cheers,
> mike
> 
> On 26-Jul-07, at 12:45 PM, Serge Egelman wrote:
> 
>>
>> We conducted a study of active phishing indicators found in current web
>> browsers by simulating spear phishing attacks.  Active phishing
>> indicators differ from passive indicators in that they interrupt the
>> user's primary task, forcing a decision to be made.  Previous studies
>> (no doubt you've read the Shared Bookmarks, right?) have shown that
>> passive indicators often go unnoticed, and when they are noticed, are
>> untrusted because users place more trust in the look and feel of the
>> destination web page.  Both IE7 and Firefox 2 include active phishing
>> warnings.
>>
>> Participants came to our lab under the guise of an online shopping
>> study.  Purchases were made from Amazon and eBay using their own
>> information.  Upon the completion of a purchase, participants were sent
>> phishing message from these sites, and were told to check their email
>> accounts to make sure that their orders were confirmed.  Participants
>> were then observed interacting with the phishing websites.  Participants
>> were placed in one of four groups: 12 users of Firefox 2
>> (http://switchersblog.com/files/firefox-phishing-protection.png), 10
>> users of IE7 who were shown the passive warning
>> (http://www.itwriting.com/images/localphishing.gif), 10 users of IE7 who
>> were shown the active phishing warning
>> (http://www.billp.com/blog/images/ie7phishing.jpg), and a control group
>> (10 users) that was shown and phishing warnings.  The purpose of the
>> control group was to determine whether participants would enter personal
>> information in the absence of a warning.
>>
>> Of the 42 participants, all but two individuals (one in the control
>> group, one in the active IE7 group) clicked at least one of the phishing
>> URLs.  The 9 participants in the control group who clicked the URLs all
>> entered login information at the phishing sites.  9 participants in the
>> passive IE7 group entered login information (1 participant obeyed the
>> warnings).  Participants ignored the passive warnings for two reasons:
>> habituation with popup messages, and lack of choices in the dialog (some
>> participants read the warnings, but since there were no options, they
>> were unsure of what to do, and thus dismissed the warnings and
>> proceeded).  Additionally, some participants were so focused on the
>> primary task (entering login information on the phishing websites) that
>> they did not notice the warnings appear in the first place.
>>
>> Among those shown the active warnings, all of the Firefox users obeyed
>> the warnings.  In the active IE7 warning group, all but two participants
>> obeyed the warnings, however there was no statistically significant
>> difference between these two groups.  Of the two who ignored the
>> warnings, one blamed habituation, and the other was fooled by the
>> message coinciding with the purchase.  This both shows that the IE7
>> warning is designed too similar to other warnings in IE (e.g. the 404
>> page), and that there will always be some users who fall for phishing
>> attacks, regardless of the strength of the warnings.
>>
>> Overall, the active warnings were effective because they interrupted the
>> users' primary tasks ("attention switch") and they forced the users to
>> make a choice in order to dismiss them ("attention maintenance").  These
>> properties were lacking in the passive indicators.  Additionally, when
>> visiting the eBay site, users were shown the EV certificate indicator
>> (i.e. the green address bar) in IE7.  None of the 42 users noticed the
>> green address bar, much less the absence of it when visiting the
>> phishing sites.  Thus, it is unreasonable to expect users to be warned
>> by the absence of an indicator.
>>
>> We also found that prior experiences with phishing had zero correlation
>> with falling for a phishing attack in our study.  One third of the
>> participants claimed to have either fallen for a phishing attack, had
>> credentials stolen, or been the victim of credit fraud in the past.
>> These individuals were equally as likely to both click on the URLs and
>> ignore the warnings as other participants.  Additionally, participants
>> who could define the term "phishing" were not anymore likely to obey (or
>> ignore) the warnings than participants who could not.  Finally, when
>> asked how they believed the phishing messages got to them, participants
>> could not answer.  They understood the websites were fraudulent, however
>> they still trusted the email messages.  This shows that there is a huge
>> disconnect with users' mental models of phishing.
>>
>> Overall we concluded that warnings within the phishing context need to
>> interrupt the user's primary task to be effective.  These warnings must
>> present clear recommendations on how to proceed.  To prevent
>> habituation, these warnings should be designed differently than dialogs
>> and need to be presented rarely (i.e. only when there's a high
>> probability of immediate danger).  Finally, warnings about high risks
>> need to fail safely, for when users do become habituated.  One
>> participant in this study who was exposed to the active IE7 warning did
>> not read it (or the options it presented), and thus clicked the red 'X'
>> in the corner to dismiss it (thus closing the browser window).  She went
>> back to the original email, clicked the link again, and again closed the
>> window.  She repeated this process five times before finally giving up,
>> and was thus prevented from giving away information to the phishing
>> website despite the fact that she never read any part of the warning.
>>
>> If you have any questions, feel free to ask.  I'm still working on the
>> paper.
>>
>>
>> serge
>>
>>
>> --/*
>> Serge Egelman
>>
>> PhD Candidate
>> Vice President for External Affairs, Graduate Student Assembly
>> Carnegie Mellon University
>>
>> Legislative Concerns Chair
>> National Association of Graduate-Professional Students
>> */
>>
> 
> 

-- 
/*
Serge Egelman

PhD Candidate
Vice President for External Affairs, Graduate Student Assembly
Carnegie Mellon University

Legislative Concerns Chair
National Association of Graduate-Professional Students
*/

Received on Thursday, 26 July 2007 17:50:16 UTC