Guideline 1

April 12, 2004

This represents a pass at analyzing the QA plans of the Web Services Description Working Group against the QA Operational Guidelines. It includes some discussion points and suggested action items for the WSDesc WG to consider.

Guideline 1. Commit to Quality Assurance in Working Group activities.
Nbr	Checkpoint	Priority	Yes	No	N/A
1.1	Define QA commitment levels for operations, specifications, and test materials.	[Priority 1]		This document represents an attempt to document our relation to the Operational Guidelines. Plan to document conformance to individual guidelines rather than strive for a specific conformance level. No commitment to operational or specification guidelines at this time.
1.2	Commit to test materials.	[Priority 2]	Charter specifies a test suite. Working Group has agreed to produce a test collection and a test assertion document, and coordinate the WSDL 2.0 Validator project at Eclipse.
1.3	Commit to complete test materials.	[Priority 3]	It is our goal to produce “at least one test case for every identifiable conformance requirement of the specification”.
1.4	Enumerate QA deliverables and expected milestones.	[Priority 1]	Timeline: · Test assertion document format complete by Last Call or shortly thereafter. · Test assertion document complete by CR. · Test suite substantially complete by end of CR. · Successful Validator execution of test suite by end of CR.
1.5	Define QA criteria for Recommendation-track advancement.	[Priority 2]	The working group will list which test cases will be used in support of the two implementations per feature CR criteria, and which optional features (e.g. certain MEPs) require zero or one implementations to exit CR.
Guideline 2. Commit to resource level for Working Group QA activities.
Nbr	Checkpoint	Priority	Yes	No	N/A
2.1	Address where and how conformance test materials will be produced.	[Priority 1]	WSDL Validator will be coordinated with Eclipse. Test collection will be maintained by W3C. The Working Group will produce the test assertion document, and ask the Working Group and the community to provide test cases.
2.2	Address QA staffing commitments.	[Priority 1]		This draft plan documents the WG’s intention to staff this activity. Working Group has not agreed to the level of commitment implied by this plan.
2.3	Request allocation of QA resources to the Working Group.	[Priority 1]	Charter specifically allows additional QA members. ACTION: Chair to formally remind member organizations of this opportunity.
Guideline 3. Synchronize QA activities with the specification milestones.
Nbr	Checkpoint	Priority	Yes	No	N/A
3.1	Synchronize the publication of QA deliverables and the specification's drafts.	[Priority 2]		Working Group has not made a commitment to publish test suites along with each specification revision.
3.2	Support specification versioning/errata in QA deliverables.	[Priority 1]	The test suite should indicate which draft of the specification is supported. Action: Test assertion document editor to make sure the test assertion framework accommodates this.
Guideline 4. Define the QA process.
Nbr	Checkpoint	Priority	Yes	No	N/A
4.1	Appoint a QA moderator.	[Priority 1]		ACTION: Chair to ask again for volunteer.
4.2	Appoint a QA task force.	[Priority 2]	QA Task force formed (though dormant) Amy and Arthur (?) are on the record as members of the QA task force.
4.3	Produce the QA Process Document.	[Priority 1]	Action: Ask Hugo to clone http://www.w3.org/2000/xp/Group/1/10/ts-contribution and update it for WSDL.
4.4	Specify means for QA-related communication.	[Priority 2]	Use WG mailing list for now.
4.5	Define branding policy details.	[Priority 3]		None
Guideline 5. Plan test materials development.
Nbr	Checkpoint	Priority	Yes	No	N/A
5.1	Define a framework for test materials development.	[Priority 2]	We will have a separate test assertion document; Arthur is the editor (28 January 2004 minutes). The test suite will consist of success and failure cases associated for each test assertion. We will correlate a sample WSDL with messages and message exchange patterns (4 March 2004 minutes) for appropriate test cases. ACTION: Test assertion document editor to create an XML format that accommodates this.
5.2	Ensure test materials are documented and usable for their intended purposes.	[Priority 1]	It is our intention to document our test suite appropriately. ACTION: Test assertion document editor to insure the framework allows test cases to be adequately categorized and documented.
5.3	Define a contribution process.	[Priority 2]		TBD, dependent upon IPR considerations.
5.4	Address license terms for submitted test materials.	[Priority 1]		Under development.
5.5	Define review procedures for submitted test materials.	[Priority 2]		Not defined at this time.
Guideline 6. Plan test materials publication.
Nbr	Checkpoint	Priority	Yes	No	N/A
6.1	Ensure a suitable repository location for test materials.	[Priority 1]	W3C CVS will hold the test suite. ACTION: QA TF to create an appropriate directory.
6.2	Define the licenses applicable to published test materials.	[Priority 1]		Under development.
6.3	Describe how and where the test materials will be published.	[Priority 2]	Working group plans to issue regular rollups of test suite changes.
6.4	Provide a conformance verification disclaimer with the test materials.	[Priority 1]	We plan to do this. ACTION: QA TF members to draft such a disclaimer.
6.5	Promote testing and the publication of test results.	[Priority 2]		No specific plan at this time.
Guideline 7. Plan the transfer of test materials to W3C if needed.
Nbr	Checkpoint	Priority	Yes	No	N/A
7.1	Perform a quality assessment of any test materials that are candidates for transfer.	[Priority 2]			No transfers planned at this time.
7.2	Identify sufficient staff resources to meet the needs of any transferred test materials.	[Priority 1]			No transfers planned at this time.
7.3	For any transferred test materials, resolve all IPR issues with the external party that produced the test materials.	[Priority 1]			No transfers planned at this time.
Guideline 8. Plan for test materials maintenance.
Nbr	Checkpoint	Priority	Yes	No	N/A
8.1	Provide for the long-term maintenance of the contribution and review procedures.	[Priority 3]		Not planned at this time.
8.2	Specify a test materials update procedure to track new specification versions/errata.	[Priority 1]		Not planned at this time.
8.3	Identify a procedure for test validity appeals.	[Priority 2]		Not planned at this time.