Draft text for section 9.3 of Note - Implementation and Testing - ACTION-121

Implementation and Testing 

Part of a Working Group's activities is developing sample code and test 
suites: 
http://www.w3.org/2005/10/Process-20051014/process.html

Sample code to demonstrate and test the WG's recommendations on display of 
security context information will be implemented in one or more web user 
agents, as extensions to them The most likely web user agents we will use 
as implementation platforms are web browsers. The sample code will be made 
available publiclly as part of the WG's deliverables. To ensure 
interoperability and generality of the recommendations, they will be 
implemented in the context of at least two web user agents. Entrance to 
Proposed Recommendations required two interoperable implementations of 
each feature of a specification. 

We are targetting three types of testing of our recommendations: 
functional testing, robustness testing, and usability testing. 
http://www.w3.org/QA/WG/2005/01/test-faq

All test development and testing is iterative. Test planning ideally 
starts when work on the specification starts. Testing planning will 
include guidelines for developing tests. Test suites are typically 
developed when the specifications are in a reasonably stable state, such 
as the first full public working draft. Test development will include test 
execution instructions. Automation of the tests will be considered but is 
unlikely, as the tests will require human visual confirmation. Clear 
descriptions of what to expect and how to judge outcome will be part of 
each test. 

Functional testing against the sample code and appropriate deployment 
configurations will verify that the recommendations can be translated to 
web user agent code, with no functional ill effects on the rest of the web 
user agent. It will show that implementations can conform to the 
recommendations, and that the specifications clearly define behaviors. 
This is also called conformance testing. 

Robustness testing will verify that the recommendations are robust against 
spoofing attacks. Existing spoofing attacks will be documented, and new 
spoofing attacks aimed directly at the recommendations (both required and 
recommended) will be developed. All of these attacks will take the form of 
web site content returned to the user agent (most typically DHTML or XML 
that a web browser GETs). 

Usability testing will verify that the recommendations provide usable 
display of security context information. The type of usability testing we 
do will depend on both the direction of our recommendations and the 
resources the team is able to tap into. While members of the Working Group 
typically develop tests, we require specific outreach in this area. One 
area for outreach is to parts of WG member organizations not specifically 
represented by active members of the WG, particularly existing usbility 
testing groups. The other area for outreach is to organizations actively 
involved in usability testing of security context information, including 
academic and industry research organizations. At a minimum, we will find 
the resources to do "low fidelity" paper usability testing on a modest 
number volunteers (10 - 20) from WG member organizations. We will pursue 
resources that allow us to do more extensive usability testing, including:

o Substantially different low fi testing for each iteration
o Broader and more representative user population participating 
o Lab testing of sample code (for example, 
http://cups.cs.cmu.edu/soups/2005/2005proceedings/p13-garfinkel.pdf) 
o Contextual or "in the wild" testing of sample code (for example, 
http://www.indiana.edu/~phishing/social-network-experiment/phishing-preprint.pdf) 

o More iterative combinations of the above, throughout the specification 
lifecycle