Re: ACTION-232 OPEN Share results from his study once he has them

Huh. That's bizarre. At first I was getting what looked like a  
screenshot of an OSX dock looking at a product called "1 Password for  
Firefox". Refreshing the page made it better.

nm!

cheers,
mike

On 26-Jul-07, at 1:52 PM, Serge Egelman wrote:

> What's wrong with the screenshot?  It shows up correctly for me....
>
> serge
>
> Mike Beltzner wrote:
>> Serge, thanks for sharing these results. Were there any insights into
>> the user's mental model when they hit a warning after expecting to
>> arrive at a legitimate website? Did you get any feeling about  
>> whether or
>> not the language used in the warnings had any effect in addition  
>> to the
>> active vs. passive nature of the warnings?
>>
>> Also, I think your link to the screenshot of Firefox2's anti-phishing
>> warning is incorrect.
>>
>> cheers,
>> mike
>>
>> On 26-Jul-07, at 12:45 PM, Serge Egelman wrote:
>>
>>>
>>> We conducted a study of active phishing indicators found in  
>>> current web
>>> browsers by simulating spear phishing attacks.  Active phishing
>>> indicators differ from passive indicators in that they interrupt the
>>> user's primary task, forcing a decision to be made.  Previous  
>>> studies
>>> (no doubt you've read the Shared Bookmarks, right?) have shown that
>>> passive indicators often go unnoticed, and when they are noticed,  
>>> are
>>> untrusted because users place more trust in the look and feel of the
>>> destination web page.  Both IE7 and Firefox 2 include active  
>>> phishing
>>> warnings.
>>>
>>> Participants came to our lab under the guise of an online shopping
>>> study.  Purchases were made from Amazon and eBay using their own
>>> information.  Upon the completion of a purchase, participants  
>>> were sent
>>> phishing message from these sites, and were told to check their  
>>> email
>>> accounts to make sure that their orders were confirmed.   
>>> Participants
>>> were then observed interacting with the phishing websites.   
>>> Participants
>>> were placed in one of four groups: 12 users of Firefox 2
>>> (http://switchersblog.com/files/firefox-phishing-protection.png), 10
>>> users of IE7 who were shown the passive warning
>>> (http://www.itwriting.com/images/localphishing.gif), 10 users of  
>>> IE7 who
>>> were shown the active phishing warning
>>> (http://www.billp.com/blog/images/ie7phishing.jpg), and a control  
>>> group
>>> (10 users) that was shown and phishing warnings.  The purpose of the
>>> control group was to determine whether participants would enter  
>>> personal
>>> information in the absence of a warning.
>>>
>>> Of the 42 participants, all but two individuals (one in the control
>>> group, one in the active IE7 group) clicked at least one of the  
>>> phishing
>>> URLs.  The 9 participants in the control group who clicked the  
>>> URLs all
>>> entered login information at the phishing sites.  9 participants  
>>> in the
>>> passive IE7 group entered login information (1 participant obeyed  
>>> the
>>> warnings).  Participants ignored the passive warnings for two  
>>> reasons:
>>> habituation with popup messages, and lack of choices in the  
>>> dialog (some
>>> participants read the warnings, but since there were no options,  
>>> they
>>> were unsure of what to do, and thus dismissed the warnings and
>>> proceeded).  Additionally, some participants were so focused on the
>>> primary task (entering login information on the phishing  
>>> websites) that
>>> they did not notice the warnings appear in the first place.
>>>
>>> Among those shown the active warnings, all of the Firefox users  
>>> obeyed
>>> the warnings.  In the active IE7 warning group, all but two  
>>> participants
>>> obeyed the warnings, however there was no statistically significant
>>> difference between these two groups.  Of the two who ignored the
>>> warnings, one blamed habituation, and the other was fooled by the
>>> message coinciding with the purchase.  This both shows that the IE7
>>> warning is designed too similar to other warnings in IE (e.g. the  
>>> 404
>>> page), and that there will always be some users who fall for  
>>> phishing
>>> attacks, regardless of the strength of the warnings.
>>>
>>> Overall, the active warnings were effective because they  
>>> interrupted the
>>> users' primary tasks ("attention switch") and they forced the  
>>> users to
>>> make a choice in order to dismiss them ("attention  
>>> maintenance").  These
>>> properties were lacking in the passive indicators.  Additionally,  
>>> when
>>> visiting the eBay site, users were shown the EV certificate  
>>> indicator
>>> (i.e. the green address bar) in IE7.  None of the 42 users  
>>> noticed the
>>> green address bar, much less the absence of it when visiting the
>>> phishing sites.  Thus, it is unreasonable to expect users to be  
>>> warned
>>> by the absence of an indicator.
>>>
>>> We also found that prior experiences with phishing had zero  
>>> correlation
>>> with falling for a phishing attack in our study.  One third of the
>>> participants claimed to have either fallen for a phishing attack,  
>>> had
>>> credentials stolen, or been the victim of credit fraud in the past.
>>> These individuals were equally as likely to both click on the  
>>> URLs and
>>> ignore the warnings as other participants.  Additionally,  
>>> participants
>>> who could define the term "phishing" were not anymore likely to  
>>> obey (or
>>> ignore) the warnings than participants who could not.  Finally, when
>>> asked how they believed the phishing messages got to them,  
>>> participants
>>> could not answer.  They understood the websites were fraudulent,  
>>> however
>>> they still trusted the email messages.  This shows that there is  
>>> a huge
>>> disconnect with users' mental models of phishing.
>>>
>>> Overall we concluded that warnings within the phishing context  
>>> need to
>>> interrupt the user's primary task to be effective.  These  
>>> warnings must
>>> present clear recommendations on how to proceed.  To prevent
>>> habituation, these warnings should be designed differently than  
>>> dialogs
>>> and need to be presented rarely (i.e. only when there's a high
>>> probability of immediate danger).  Finally, warnings about high  
>>> risks
>>> need to fail safely, for when users do become habituated.  One
>>> participant in this study who was exposed to the active IE7  
>>> warning did
>>> not read it (or the options it presented), and thus clicked the  
>>> red 'X'
>>> in the corner to dismiss it (thus closing the browser window).   
>>> She went
>>> back to the original email, clicked the link again, and again  
>>> closed the
>>> window.  She repeated this process five times before finally  
>>> giving up,
>>> and was thus prevented from giving away information to the phishing
>>> website despite the fact that she never read any part of the  
>>> warning.
>>>
>>> If you have any questions, feel free to ask.  I'm still working  
>>> on the
>>> paper.
>>>
>>>
>>> serge
>>>
>>>
>>> --/*
>>> Serge Egelman
>>>
>>> PhD Candidate
>>> Vice President for External Affairs, Graduate Student Assembly
>>> Carnegie Mellon University
>>>
>>> Legislative Concerns Chair
>>> National Association of Graduate-Professional Students
>>> */
>>>
>>
>
> -- 
> /*
> Serge Egelman
>
> PhD Candidate
> Vice President for External Affairs, Graduate Student Assembly
> Carnegie Mellon University
>
> Legislative Concerns Chair
> National Association of Graduate-Professional Students
> */

Received on Thursday, 26 July 2007 18:03:28 UTC