Re: ACTION-335 logotypes and ISSUE-96 discussion

I worry about over-reliance on prior user studies without delving into the
details. I know I've seen studies that show particular implementations to
have particular flaws that rendered the particular implementation
ineffective. I don't know that I've seen studies that say the same thing
about the entire concept.

I would say that if we agreed on a set of conditions that we wanted to test
for "success", given another set of controls (including implementations),
and that experimental setup had already been tested, then we should not
waste time re-running that same experiment. However, if there are sufficient
changes (either in implementations, concepts, base assumptions, or overall
experimental design) then I think we have to take that into consideration.

In short, I don't want to waste people's time by re-running experiments
needlessly, but if we see problems with prior experiments I don't want to
rely on their results in an overly-broad manner.

-Ian

On Nov 12, 2007 3:55 PM, Serge Egelman <egelman@cs.cmu.edu> wrote:

> I agree completely.  However, if testing or prior work show that the
> perfect world scenario (where all good sites use EV certs) is completely
> flawed (i.e. EV logos will never be noticed by users, be susceptible to
> spoofing, etc.), then there's little value in testing the realistic
> scenario.  Agreed?
>
> serge
>
> Ian Fette wrote:
> > This action (ACTION-335) was to provide discussion topics for ISSUE-96.
> > I only really have one point, and I will try to state it more clearly
> > than at the meeting.
> >
> > To me, the effectiveness of any of the logotype proposals (or the EV
> > proposals, for that matter) depends greatly upon the adoption of these
> > technologies by sites. We can do really cool flashy things when we get
> > an EV cert, or an EV-cert with a logo, but right now the only two sites
> > I can find using an EV cert are PayPal and VeriSign. Therefore, I wonder
> > how habituated people would become in practice, if they never (or
> > rarely) saw the EV/logotype interface stuff in use.
> >
> > My proposal is that any usability testing of the EV and/or logotype
> > things in the spec not only reflect how users would behave in a land
> > where everyone is using EV-certs and life is happy, but rather also test
> > a more realistic case. That is, look at what the adoption is presently
> > and/or what we can reasonably expect it to be at time of last call, and
> > do usability testing in an environment that reflects that adoption rate
> > - i.e. some percentage of sites using EV certs, some percentage also
> > using logos, and another percentage still using "normal" SSL certs. My
> > worry is that we may be thinking "EV certs will solve X,Y, and Z", but
> > that may only be the case if users are used to seeing them on the
> > majority of sites, and should that not end up being the case, we need to
> > look at the usability and benefit in that scenario as well.
> >
> > I think this is what the ACTION wanted, i.e. for me to state this point
> > more explicitly. I am going to therefore assume that my work on this
> > action is complete, unless I hear otherwise.
> >
> > -Ian
>
> --
> /*
> PhD Candidate
> Vice President for External Affairs, Graduate Student Assembly
> Carnegie Mellon University
>
> Legislative Concerns Chair
> National Association of Graduate-Professional Students
> */
>

Received on Tuesday, 13 November 2007 00:51:06 UTC