RE: ISSUE-117 (serge): Eliminating Faulty Recommendations [All]

OK, I'm cc-ing the group list.  Thanks, Mike

  _____  

From: Mary Ellen Zurko [mailto:Mary_Ellen_Zurko@notesdev.ibm.com] 
Sent: Monday, November 12, 2007 7:01 AM
To: McCormick, Mike
Subject: RE: ISSUE-117 (serge): Eliminating Faulty Recommendations [All]



You should start that discussion on the group's list. 

          Mez



From: 	<michael.mccormick@wellsfargo.com> 
To: 	<Mary_Ellen_Zurko@notesdev.ibm.com> 
Date: 	11/09/2007 11:33 PM 
Subject: 	RE: ISSUE-117 (serge): Eliminating Faulty
Recommendations [All]

  _____  


Criteria 2, at least as phrased below, concerns me.  I don't feel WSC
should be constrained from making a recommendation just because a
particular community may resist adopting it.  Our guidance on favicons
is a case in point.  I'm skeptical browsers will adopt that
recommendation any time soon but it's still the right thing to do.  If
browser manufacturers could always be counted on to do the right things
for security on their own, then initiatives like WSC would be less
necessary.  Criteria 2 could also reinforce a perception among some
skeptics that W3C is beholden to certain web technology vendors and
gives their needs priority over those of other industries or the broader
user community.  Just my $.02.  Mike


  _____  

From: public-wsc-wg-request@w3.org [mailto:public-wsc-wg-request@w3.org
<mailto:public-wsc-wg-request@w3.org> ] On Behalf Of Mary Ellen Zurko
Sent: Friday, November 09, 2007 2:56 PM
To: Web Security Context Working Group WG
Subject: Re: ISSUE-117 (serge): Eliminating Faulty Recommendations [All]


Our discussions on baseline success criteria and ISSUE-112 at the f2f
provided the input I needed to respond to this. 
http://www.w3.org/2006/WSC/wiki/SuccessBaseline
<http://www.w3.org/2006/WSC/wiki/SuccessBaseline> 
[also see the minutes from November 6 on the topic of  ISSUE-112,
currently members only] 

I would argue to eliminate any recommendation that we believe we could
not get buy in for (and that we did not believe in the future of uptake
of) from the appropriate community (browsers, web app developers, web
site administrators, users) (see criteria 2). 

I would also argue to eliminate any recommendation that neither captured
current best practice (criteria 3) nor had WG consensus that it would be
demonstrably better at aiding trust decisions than the state before the
WG started (criteria 4). 

The last line of this issue seems to ask about the place of prior user
studies and literature in this process. I see them feeding into criteria
4. For any of our recommendations, anyone can challenge whether or not
they help in aiding trust decisions. Prior user studies and literature
may be the reason why (or part of the reason why). We discuss it,
including any other information or data on the topic, then see what
group consensus is. Other sorts of data may be brought to bear on the
topic; see 
http://www.w3.org/TR/wsc-usecases/#process
<http://www.w3.org/TR/wsc-usecases/#process> 

I bring up the ISSUE-112 here as well because I do not want anyone
wasting time doing any user studies if the results will be discounted by
the group during discussions. That would be unfair and disrespectful. My
advice is that for any user study done specifically for this group, we
specify ahead of time what we're doing, what sort of outcomes might be
expected, and how that should influence our recommendation. We then
discuss _that_ and get group consensus on the trajectory and impact of a
user study before actually running it. If we can run this process with
something modest soon, it can helpfully provide input to anything more
resource intensive we do later, and see if that's a reasonable way to
integrate them into our work. 

As a side note, since I consider myself an actual expert (for some value
of expert) on the topic of usable security, I'm likely to want to read
the data on prior user studies and literature that people cite. I've
tried to stay on top of our bookmarks, and will continue to try to stay
on top of citations used in discussion. I find it deeply irksome when
there's a reference that I can't get to (the ACM Portal being the
canonical example for me). That doesn't mean that citations there won't
have impact, just the way deployment or product experience that is based
on data not directly available to all of us have impact. It means it
will be subject to the same sort of engagement by wg members, to try to
understand and reason about it in the WG context. 

Any other takers on this issue before I put it on a meeting agenda?
Additional ideas, expectations, suggestions, assumptions, presumptions? 



From: 	Web Security Context Working Group Issue Tracker
<sysbot+tracker@w3.org> 
To: 	public-wsc-wg@w3.org 
Date: 	10/08/2007 12:56 PM 
Subject: 	ISSUE-117 (serge): Eliminating Faulty Recommendations
[All]



  _____  






ISSUE-117 (serge): Eliminating Faulty Recommendations [All]

http://www.w3.org/2006/WSC/track/issues/
<http://www.w3.org/2006/WSC/track/issues/> 

Raised by: Serge Egelman
On product: All

At what point can we say that a recommendation is unlikely to work and
should be removed from consideration?

For some of these we obviously need user studies to see how effective
the techniques are.  However, if prior user studies and literature have
already tested similar concepts, it would be a waste of our time and
resources to test them again.