Re: ISSUE-117 (serge): Eliminating Faulty Recommendations [All]

Yes. I would add teeing up an agenda item with me to have the group 
discussion on it, when you're ready. 

          Mez





From:
Maritza Johnson <maritzaj@cs.columbia.edu>
To:
Mary Ellen Zurko <Mary_Ellen_Zurko@notesdev.ibm.com>
Cc:
Web Security Context Working Group WG <public-wsc-wg@w3.org>
Date:
11/20/2007 10:16 AM
Subject:
Re: ISSUE-117 (serge): Eliminating Faulty Recommendations [All]




Late apologies for missing the call on this ... I've been running a user 
study for another project the past two weeks and my schedule's been 
dictated by when people can show up for sessions (I know, lame excuse, but 
it's tough to recruit participants). The work is related to our group and 
I plan to share the results as soon as I hear back from Steve about when 
and how much I can say.


I bring up the ISSUE-112 here as well because I do not want anyone wasting 
time doing any user studies if the results will be discounted by the group 
during discussions. That would be unfair and disrespectful. My advice is 
that for any user study done specifically for this group, we specify ahead 
of time what we're doing, what sort of outcomes might be expected, and how 
that should influence our recommendation. We then discuss _that_ and get 
group consensus on the trajectory and impact of a user study before 
actually running it. If we can run this process with something modest 
soon, it can helpfully provide input to anything more resource intensive 
we do later, and see if that's a reasonable way to integrate them into our 
work. 


Reading over the minutes about this issue again, I noticed Serge has an 
action item to begin going through the recommendations, I guess to do a 
more thorough job of what we started in July, maybe to produce tangible 
action items out of the process?


In agreement with what Mez said above and in response to Tim's followup on 
my comments about Browser Lockdown <
http://www.w3.org/2006/WSC/track/actions/305 >, I've given myself an 
action item <http://www.w3.org/2006/WSC/track/actions/345> to come up with 
a user study design that will begin to evaluate the unknowns for the 
recommendation.

I'll write up a study that will mostly consist of a questionnaire with the 
goal of collecting data on whether or not users can be relied on the 
choose the right mode given a certain task. It will also be designed to 
gather feedback on how users might react to being blocked from accessing a 
site. 

I'll address the points Mez has made above the best I can and post it to 
the list in hopes of generating useful discussion on refining the study.

Mez -- is this what you had in mind for moving forward?


-- Maritza