Re: ACTION-301: Usability review of Identity Signal

On 10/26/07, Mary Ellen Zurko <Mary_Ellen_Zurko@notesdev.ibm.com> wrote:
>
> > I appreciate that "help users understand the identity of sites they
> > interact with" is a harder testing problem than "prevent phishing
> > attacks" and I don't actually have a good methodology suggestion.  An
>
> I don't see why it is (and I expect kind and informative responses to
> naivete :-). The testing of understandability of visual icons goes much
> further back than usability testing around user attacks. I would expect that
> kind of UT would be the most appropriate.
>         Mez

I agree with Mez.  It is actually easier to test if the scheme helps
"users understand the identity of sites they interact with" than to
test if it prevents phishing attacks.

To do this, you need to define what you mean by "understanding
identity".  What exactly do you want users to know?  E.g. "when a user
visits the Bank X website, they understand that they are at Bank X and
not Y", or "when they visit site A that does not have an EV
certificate they understand that a third party has not verified the
identity of the site".  Your standard might be higher e.g. "they might
be suspicious" in some circumstances or be able to verify the identity
in a phishing attack that spoofs Larry (I know this is not your goal).
 Once you define the goals, we can ask users to use the interface and
then test them or interview them to see if your goals were met.

We can do this in a lab, by distributing the client to users and then
interviewing them, or you could instrument the client. Obviously, you
can get more accurate answers to behavior questions (e.g. do users
discover Larry on their own?) if you have a long term study with an
instrumented client.  However, if you have questions about what users
*understand*, there is nothing that beats the kind of data you can get
by showing users the interface and interviewing them face to face.
Computer scientists really discount the value of this methodology, and
I think our designs suffer for it.

Rachna

Received on Friday, 26 October 2007 16:34:48 UTC