RE: "Average User" / CMU Survey

Michael (and Simson, whereever you are :-), thanks for pointing this out. 

Michael, will you add but the reference and your bullet commentary to our 
shared bookmarks area:

Also, can you fold this in to Martiza's description of the user for our 
Note? Maritza, where in the wiki are you putting that? 

The percentages of those that always pay attention to a warning and those 
that never pay attention to a warning remind me of the percentages in our 
Notes ECL study. Non trivial at both ends, but neither absolute.

          Mez

Mary Ellen Zurko, STSM, IBM Lotus CTO Office       (t/l 333-6389)
Lotus/WPLC Security Strategy and Patent Innovation Architect




<michael.mccormick@wellsfargo.com> 
Sent by: public-wsc-wg-request@w3.org
01/03/2007 05:07 PM

To
<brad@tellme.com>
cc
<public-wsc-wg@w3.org>
Subject
RE: "Average User" / CMU Survey






Agreed.  I wish they'd surveyed a larger, more diverse population sample 
(they studied only 20 people, all under age 45) but their methodology 
seems well thought out and the results are certainly suggestive.  I 
suspect adding users over 45 would have skewed the results even more 
toward the "dumb user" end of the spectrum.  Mike
Michael McCormick, CISSP 
Lead Architect, Information Security 
This message may contain confidential and/or privileged information.  If 
you are not the addressee or authorized to receive this for the addressee, 
you must not use, copy, disclose, or take any action based on this message 
or any information herein.  If you have received this message in error, 
please advise the sender immediately by reply e-mail and delete this 
message.  Thank you for your cooperation.
 

From: Brad Porter [mailto:brad@tellme.com] 
Sent: Wednesday, January 03, 2007 3:55 PM
To: McCormick, Mike
Cc: public-wsc-wg@w3.org
Subject: Re: "Average User" / CMU Survey

This article is terrific.  We should still be careful about extrapolating 
an "average user" from this given their caveat:
Given this small and non-representative sample, we can¡¦t
extrapolate prevalence of beliefs to the general population. We
purposefully selected participants who were more naive than the
average, in order to understand how those without a good
understanding of security make sense of Internet risks.
That said, the user demographic studied may be exactly our target 
audience.

--Brad

michael.mccormick@wellsfargo.com wrote: 
There's been recent discussion (most recently in the "Notes section - 
Design Principles" thread) about how security savvy the "average" web 
user" is.
Simson Garfinkel of Harvard kindly drew my attention to some excellent 
work done by Carnegie Mellon last year in this area.  See 
http://cups.cs.cmu.edu/soups/2006/proceedings/p79_downs.pdf for details.
CMU used a pretty rigorous methodology to assess the web security know-how 
of users drawn from a random cross section of the Pittsburg PA population. 
 Users were questioned & observed while responding to various simulated 
possible phishing scenarios.  Browsers used were MSIE, Firefox, Netscape, 
& Safari.  Their report will be very valuable to the work of WSC -- I urge 
you all to take a look.
Some relevant highlights: 
Most participants [85%] had seen lock images on a web site, and knew that 
this was meant to signify security, but most had only a limited 
understanding of what how to interpret locks, e.g., ¡§I think that it means 
secured, it symbolizes some kind of security, somehow.¡¨  Few knew that the 
lock icon in the chrome (i.e., in the browser¡¦s border rather than the 
page content) indicated that the web site was using encryption or that 
they could click on the lock to examine the certificate. Indeed, only 40% 
of those who were aware of the lock realized that the lock had to be 
within the chrome of the browser. 
Only about a third [35%] had noticed a distinction between "http://" and "
https://" URLs.  Of those some did not think that the ¡§s¡¨ indicated 
anything. But those who were aware of the security connotation of this cue 
tended to take it as a fairly reliable indication that it is safe to enter 
information.  For those people this extra security was often enough to get 
them beyond their initial trepidations about sharing sensitive 
information, e.g., ¡§I feel funny about putting my credit card number in, 
but they say it is a secure server and some of them say ¡¥https¡¦ and 
someone said that it means it¡¦s a secure server.¡¨ 
About half [55%] had noticed a URL that was not what they expected or 
looked strange. For some, this was a reason to be wary of the website. For 
others, it was an annoyance, but no cause for suspicion.  The other half 
[45%} appeared to completely ignore the address bar and never noticed even 
the most suspicious URLs. 
Participants appeared to be especially uncertain what to make of 
certificates.  Many respondents specifically said that they did not know 
what certificates were, and made inferences about how to respond to any 
"mysterious message" mentioning certificates. Some inferred that 
certificates were a "just a formality".  Some used previous experience as 
their basis for ignoring it, e.g., ¡§I have no idea [what it means], 
because it¡¦s saying something about a trusted website or the certificate 
hasn¡¦t, but I think I¡¦ve seen it on websites that I thought were 
trustworthy.¡¨ 
Almost half [42%] recognized the self-signed certificate warning message 
as one they'd seen before.  A third [32%] always ignored this warning, a 
fourth [26%] consistently avoided entering sites when this warning was 
displayed, and the rest responded inconsistently. 
When asked about warnings generally, only about half of participants 
recalled ever having seen a warning before trying to visit a web site. 
Their recollections of what they were warned about were sometimes vague, 
e.g., ¡§sometimes they say cookies and all that,¡¨ or uncertain, e.g., 
¡§Yeah, like the certificate has expired. I don¡¦t actually know what that 
means.¡¨ When they remembered warnings about security, they often dismissed 
them with logical reasoning, e.g., ¡§Oh yeah, I have [seen warnings], but 
funny thing is I get them when I visit my [school] websites, so I get told 
that this may not be secure or something, but it¡¦s my school website so I 
feel pretty good about it.¡¨ 
Only half of participants had heard the term "phishing".  The other half 
couldn't guess what it meant.  Most participants had heard the term 
"spyware" but a number of those believed it was something good that 
protects one's computer from spies. 
Michael McCormick, CISSP 
Lead Architect, Information Security 
Wells Fargo Bank 
255 Second Avenue South 
MAC N9301-01J 
Minneapolis MN 55479 
(ƒn      612-667-9227 (desk)             7       612-667-7037 (fax) 
(       612-590-1437 (cell)             J       
michael.mccormick@wellsfargo.com (AIM) 
2       612-621-1318 (pager)            *       
michael.mccormick@wellsfargo.com 
¡§THESE OPINIONS ARE STRICTLY MY OWN AND NOT NECESSARILY THOSE OF WELLS 
FARGO" 
This message may contain confidential and/or privileged information.  If 
you are not the addressee or authorized to receive this for the addressee, 
you must not use, copy, disclose, or take any action based on this message 
or any information herein.  If you have received this message in error, 
please advise the sender immediately by reply e-mail and delete this 
message.  Thank you for your cooperation.