RE: Some studies on the visibility of EV sites.

> From: Serge Egelman [mailto:egelman@cs.cmu.edu] 

> That study was not performed under field conditions.  A field 
> study involves observing participants in their natural 
> setting.  Calling them up and walking them through a series 
> of artificial tasks is not a field study.  That's a 
> laboratory study, it's a laboratory study being performed in the home.

I was referring to the shopping cart studies as field conditions. 


At the moment we know that it is impossible to effectively and reliably
explain the security of the padlock icon to the user under any
circumstances. I was unable to demonstrate it to Danny W. sitting next
to him. The interviews demonstrate that we can cause the user to be
confident that they have understood the instructions. Given where we
start from that is an advance. 


> > Now that is not the same as demonstrating that the users 
> will make the 
> > right choice when faced with an attack. But I don't know how to 
> > measure that accurately. I don't think anyone else does either.
> > 
> 
> You could start by attacking the users and observing what 
> they do.  Of course, telling them ahead of time that the 
> study is about security and that they will be attacked is 
> going to confound your results.  I would hope that everyone 
> can agree on this very basic point.

Its kind of hard to bring people into a lab situation without them
making any assumptions as to the purpose of the study. If you tell the
users that they are looking at a prototype you confound the results.
They are now likely to interpret failures or errors as being due to the
lab environment.

We don't usually take users into an lab and attack them in ways that is
likely to result in real harm.

And I don't think we can do attacks in the field very easily and stay
within ethical and legal boundaries. We can observe actual responses to
attacks.


> > The scenario I care about is the one in which we have a user who
> > receives a phishing bait email in their inbox, is 
> suspicious but follows
> > the link. In that scenario the user is primed to be 
> security aware by
> > definition.
> 
> No, you clearly don't understand the definition of priming.  
> Priming is 
> when the subject has been tipped off as to the purpose of the 
> study and 
> is therefore likely to exhibit behaviors that will differ 
> from how they 
> would normally behave

No, I have a well documented dislike of the use of specialist
vocabularies, particularly the attempt to use control of language to
control the terms of debate, whose credentials are recognized etc.

I understand the term to have a wider meaning than you attempt to insist
on here. Priming means putting an object in a state where it is
predisposed to a certain action by analogy to priming a gun to fire.

In standard phishing bait the attacker deliberately attempts to create a
security concern. 


> In this case, the subjects have been 
> primed for security and the results of the study will be completely 
> bogus (assuming the purpose was to examine how many people notice the 
> indicators under natural conditions).

How would you suggest going about that?


> > We do have a large number of users who are suspicious when they get
> > these emails. The problem is that there are two possible outcomes
> > (caught / not caught) and the attacker can deliberately confuse the
> > user.
> > 
> > I do not expect to get perfect results. A bank told me that 15% of
> > customers who are suspicious enough to call the bank then 
> go ahead and
> > give their details AFTER being warned that it was a scam (!)
> 
> Doesn't that cause great concern regarding the number of 
> users who don't call the bank?

Say the cost of the telephone call is $15.
Say the typical loss, without recovery is $400 and 3% of phished cards
are used, the expected loss is $12

So the customer who calls you to ask is possibly causing a greater loss
than the one who gives their card #.

Recovery shifts the costs somewhat, if the recovery attempt costs you
$300 and the recovery rate is 70% that raises your loss by $90 and it's
a wash.


The banks are not that concerned about the Soc General affair either.
Desipte the fact that the criminals are going to be looking to duplicate
the results on purpose.

> The laboratory study wasn't examining whether this system 
> would improve 
> Bank of America's bottom line.  The point of the study was to examine 
> how much security it really provided.  Clearly whether it 
> provides real 
> security and whether it's a cost effective measure are two completely 
> different questions.

As long as it's the bank that's bearing the loss that sounds fair enough
to me.

The consumer concern should in fact be lack of confidence.

Received on Thursday, 6 March 2008 19:20:03 UTC