RE: Draft text for section 9.3 of Note - Implementation and Testing - ACTION-121

Done.
 
See:
 
http://www.w3.org/2006/WSC/drafts/note/#usability-testing
 
Tyler


________________________________

	From: public-wsc-wg-request@w3.org
[mailto:public-wsc-wg-request@w3.org] On Behalf Of Mary Ellen Zurko
	Sent: Wednesday, February 07, 2007 7:53 AM
	To: public-wsc-wg@w3.org
	Cc: Mary Ellen Zurko
	Subject: Draft text for section 9.3 of Note - Implementation and
Testing - ACTION-121
	
	

	Implementation and Testing 
	
	Part of a Working Group's activities is developing sample code
and test suites: 
	http://www.w3.org/2005/10/Process-20051014/process.html
	
	Sample code to demonstrate and test the WG's recommendations on
display of security context information will be implemented in one or
more web user agents, as extensions to them The most likely web user
agents we will use as implementation platforms are web browsers. The
sample code will be made available publiclly as part of the WG's
deliverables. To ensure interoperability and generality of the
recommendations, they will be implemented in the context of at least two
web user agents. Entrance to Proposed Recommendations required two
interoperable implementations of each feature of a specification. 
	
	We are targetting three types of testing of our recommendations:
functional testing, robustness testing, and usability testing. 
	http://www.w3.org/QA/WG/2005/01/test-faq
	
	All test development and testing is iterative. Test planning
ideally starts when work on the specification starts. Testing planning
will include guidelines for developing tests. Test suites are typically
developed when the specifications are in a reasonably stable state, such
as the first full public working draft. Test development will include
test execution instructions. Automation of the tests will be considered
but is unlikely, as the tests will require human visual confirmation.
Clear descriptions of what to expect and how to judge outcome will be
part of each test. 
	
	Functional testing against the sample code and appropriate
deployment configurations will verify that the recommendations can be
translated to web user agent code, with no functional ill effects on the
rest of the web user agent. It will show that implementations can
conform to the recommendations, and that the specifications clearly
define behaviors. This is also called conformance testing. 
	
	Robustness testing will verify that the recommendations are
robust against spoofing attacks. Existing spoofing attacks will be
documented, and new spoofing attacks aimed directly at the
recommendations (both required and recommended) will be developed. All
of these attacks will take the form of web site content returned to the
user agent (most typically DHTML or XML that a web browser GETs). 
	
	Usability testing will verify that the recommendations provide
usable display of security context information. The type of usability
testing we do will depend on both the direction of our recommendations
and the resources the team is able to tap into. While members of the
Working Group typically develop tests, we require specific outreach in
this area. One area for outreach is to parts of WG member organizations
not specifically represented by active members of the WG, particularly
existing usbility testing groups. The other area for outreach is to
organizations actively involved in usability testing of security context
information, including academic and industry research organizations. At
a minimum, we will find the resources to do "low fidelity" paper
usability testing on a modest number volunteers (10 - 20) from WG member
organizations. We will pursue resources that allow us to do more
extensive usability testing, including:
	
	o Substantially different low fi testing for each iteration
	o Broader and more representative user population participating 
	o Lab testing of sample code (for example,
http://cups.cs.cmu.edu/soups/2005/2005proceedings/p13-garfinkel.pdf) 
	o Contextual or "in the wild" testing of sample code (for
example,
http://www.indiana.edu/~phishing/social-network-experiment/phishing-prep
rint.pdf) 
	o More iterative combinations of the above, throughout the
specification lifecycle