W3C

Checklist of Checkpoints for "QA Framework: Operational Guidelines"

This version:
http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/qaframe-ops-checklist
Completed by:
Lynne Rosenthal and Tim Boland, CSS QA Moderator

In general, CSS incorporates many QA practices, but does not do exactly what OpsGL requires. Thus, many NO or partial YES answers. All of the QA practices were implemented without OpsGL, except for appointing a QA Moderator - whose role and activities isn't explicitly defined. CSS has a Roadmap document that maps out their deliverables and milestones, including tests. They Roadmap shows their commitment to developing tests for each specification and its revisions. Tests and test tools are made public. From their test page, there is a link to the QA A

Checklist table

Guideline 1. Commit to Quality Assurance in Working Group activities.
Nbr Checkpoint Priority Yes No N/A Comments
1.1

Define QA commitment levels for operations, specifications, and test materials.

 [Priority 1]    N   T-difficult to implement. Goal of Roadmap to include test development. L- For Ops and Test, Roadmap=evidence of commitment
1.2

Commit to test materials.

 [Priority 2]  Y     T-partially implemented. Informal agreement to have tests when exit CR. Separate Test web page
1.3

Commit to complete test materials.

 [Priority 3]  Y     T-parially implemented. Goal of Roadmap.
1.4

Enumerate QA deliverables and expected milestones.

 [Priority 1]  Y     T-informally understoodL-when think QA focused on tests, but CSS also provides tools, templates, etc.
1.5

Define QA criteria for Recommendation-track advancement.

 [Priority 2]  Y     T-partially implemented. Tests to exit CR

Guideline 2. Commit to resource level for Working Group QA activities.
Nbr Checkpoint Priority Yes No N/A
2.1

Address where and how conformance test materials will be produced.

 [Priority 1]  Y     T-identify person to develop tests. Plan to have person use CVS
2.2

Address QA staffing commitments.

 [Priority 1]  Y     T-partially implemented. QA Moderator appointed, Test Suite coordinator.
2.3

Request allocation of QA resources to the Working Group.

 [Priority 1]    N   T-Discussed. Companies not have additional resources

Guideline 3. Synchronize QA activities with the specification milestones.
Nbr Checkpoint Priority Yes No N/A
3.1

Synchronize the publication of QA deliverables and the specification's drafts.

 [Priority 2]  Y     T-not implemented, but goal of Roadmap that tests availalbe < 3months after draft. Committed to maintaining quality and relevance of test
3.2

Support specification versioning/errata in QA deliverables.

 [Priority 1]  Y    

Guideline 4. Define the QA process.
Nbr Checkpoint Priority Yes No N/A
4.1

Appoint a QA moderator.

 [Priority 1]  Y    
4.2

Appoint a QA task force.

 [Priority 2]    N   QA Moderator and Test Coordinators are appointed, others on an as-need basis
4.3

Produce the QA Process Document.

 [Priority 1]  Y     T-partially implemented. Draft produced
4.4

Specify means for QA-related communication.

 [Priority 2]  Y    
4.5

Define branding policy details.

 [Priority 3]    N   T-some discussion, no decision. CSS Icon does exist

Guideline 5. Plan test materials development.
Nbr Checkpoint Priority Yes No N/A
5.1

Define a framework for test materials development.

 [Priority 2]  Y     informal agreement on process, test authoring instructions
5.2

Ensure test materials are documented and usable for their intended purposes.

 [Priority 1]  Y    
5.3

Define a contribution process.

 [Priority 2]  Y     T-partially implemented. plan to define a contribution process in 2004
5.4

Address license terms for submitted test materials.

 [Priority 1]    N   T-no decision reached
5.5

Define review procedures for submitted test materials.

 [Priority 2]  Y     T-partially implemented. no formal process, but WG reviews all tests, submitor notified, any disputes are handled

Guideline 6. Plan test materials publication.
Nbr Checkpoint Priority Yes No N/A
6.1

Ensure a suitable repository location for test materials.

 [Priority 1]  Y     L-publicly available. CVS used in development
6.2

Define the licenses applicable to published test materials.

 [Priority 1]    N   T-no decision yet

L- didn't see a license statement on the test suite
6.3

Describe how and where the test materials will be published.

 [Priority 2]  Y     published and available from

www.w3.org/Style/CSS/Test
6.4

Provide a conformance verification disclaimer with the test materials.

 [Priority 1]    N  
6.5

Promote testing and the publication of test results.

 [Priority 2]    N   T-testing promoted via discussions. Plans to make Implementation report templates public.

Guideline 7. Plan the transfer of test materials to W3C if needed.
Nbr Checkpoint Priority Yes No N/A
7.1

Perform a quality assessment of any test materials that are candidates for transfer.

 [Priority 2]    N   T-CSS would conduct a quality review if transfer was planned
7.2

Identify sufficient staff resources to meet the needs of any transferred test materials.

 [Priority 1]      N/A
7.3

For any transferred test materials, resolve all IPR issues with the external party that produced the test materials.

 [Priority 1]      N/A

Guideline 8. Plan for test materials maintenance.
Nbr Checkpoint Priority Yes No N/A
8.1

Provide for the long-term maintenance of the contribution and review procedures.

 [Priority 3]  Y     T-parially implemented. Likely that a written procedure will be defined.
8.2

Specify a test materials update procedure to track new specification versions/errata.

 [Priority 1]  Y    
8.3

Identify a procedure for test validity appeals.

 [Priority 2]  Y     T-done informally. Plans to write procedure in 2004

References

QAF-OPS
QA Framework: Operational Guidelines, L. Henderson, D. Hazaël-Massieux, L. Rosenthal, Eds., W3C Candidate Recommendation, September 2003, available at http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/.