Re: Draft text for section 9.3 of Note - Implementation and Testing - ACTION-121

On 2007-02-12 15:48:51 -0600, Close, Tyler J. wrote:

> Done.
>  
> See:
>  
> http://www.w3.org/2006/WSC/drafts/note/#usability-testing

I don't think this takes the my comments here into account:

  http://lists.w3.org/Archives/Public/public-wsc-wg/2007Feb/0034.html

Do you want me to ope an issue, or do you want to take a first stab
at merging those remarks in?

> 	From: public-wsc-wg-request@w3.org
> [mailto:public-wsc-wg-request@w3.org] On Behalf Of Mary Ellen Zurko
> 	Sent: Wednesday, February 07, 2007 7:53 AM
> 	To: public-wsc-wg@w3.org
> 	Cc: Mary Ellen Zurko
> 	Subject: Draft text for section 9.3 of Note - Implementation and
> Testing - ACTION-121
> 	
> 	
> 
> 	Implementation and Testing 
> 	
> 	Part of a Working Group's activities is developing sample code
> and test suites: 
> 	http://www.w3.org/2005/10/Process-20051014/process.html
> 	
> 	Sample code to demonstrate and test the WG's recommendations on
> display of security context information will be implemented in one or
> more web user agents, as extensions to them The most likely web user
> agents we will use as implementation platforms are web browsers. The
> sample code will be made available publiclly as part of the WG's
> deliverables. To ensure interoperability and generality of the
> recommendations, they will be implemented in the context of at least two
> web user agents. Entrance to Proposed Recommendations required two
> interoperable implementations of each feature of a specification. 
> 	
> 	We are targetting three types of testing of our recommendations:
> functional testing, robustness testing, and usability testing. 
> 	http://www.w3.org/QA/WG/2005/01/test-faq
> 	
> 	All test development and testing is iterative. Test planning
> ideally starts when work on the specification starts. Testing planning
> will include guidelines for developing tests. Test suites are typically
> developed when the specifications are in a reasonably stable state, such
> as the first full public working draft. Test development will include
> test execution instructions. Automation of the tests will be considered
> but is unlikely, as the tests will require human visual confirmation.
> Clear descriptions of what to expect and how to judge outcome will be
> part of each test. 
> 	
> 	Functional testing against the sample code and appropriate
> deployment configurations will verify that the recommendations can be
> translated to web user agent code, with no functional ill effects on the
> rest of the web user agent. It will show that implementations can
> conform to the recommendations, and that the specifications clearly
> define behaviors. This is also called conformance testing. 
> 	
> 	Robustness testing will verify that the recommendations are
> robust against spoofing attacks. Existing spoofing attacks will be
> documented, and new spoofing attacks aimed directly at the
> recommendations (both required and recommended) will be developed. All
> of these attacks will take the form of web site content returned to the
> user agent (most typically DHTML or XML that a web browser GETs). 
> 	
> 	Usability testing will verify that the recommendations provide
> usable display of security context information. The type of usability
> testing we do will depend on both the direction of our recommendations
> and the resources the team is able to tap into. While members of the
> Working Group typically develop tests, we require specific outreach in
> this area. One area for outreach is to parts of WG member organizations
> not specifically represented by active members of the WG, particularly
> existing usbility testing groups. The other area for outreach is to
> organizations actively involved in usability testing of security context
> information, including academic and industry research organizations. At
> a minimum, we will find the resources to do "low fidelity" paper
> usability testing on a modest number volunteers (10 - 20) from WG member
> organizations. We will pursue resources that allow us to do more
> extensive usability testing, including:
> 	
> 	o Substantially different low fi testing for each iteration
> 	o Broader and more representative user population participating 
> 	o Lab testing of sample code (for example,
> http://cups.cs.cmu.edu/soups/2005/2005proceedings/p13-garfinkel.pdf) 
> 	o Contextual or "in the wild" testing of sample code (for
> example,
> http://www.indiana.edu/~phishing/social-network-experiment/phishing-prep
> rint.pdf) 
> 	o More iterative combinations of the above, throughout the
> specification lifecycle 
> 	
> 	
> 	
> 	
> 

-- 
Thomas Roessler, W3C  <tlr@w3.org>

Received on Tuesday, 13 February 2007 08:26:00 UTC