My action items in order from Mar 1 Minutes

Action Item 1:

Guideline 1:

NOTE: This is from "abstract" publicly available when considering whether to order ISO document:

"ISO/TS 16071:2002 provides guidance on the design of accessible (work, home, education) software. It covers issues associated with designing accessible software for people with the widest range of visual, hearing, motor and cognitive abilities, including those who are elderly and temporarily disabled. ISO/TS 16071:2002 addresses software considerations for accessibility that complement general design for usability covered by ISO 9241-10 to ISO 9241-17 and ISO 13407.

ISO/TS 16071:2002 addresses the accessibility of interactive systems. It addresses a wide range of solutions, including office applications, web pages and multimedia. It does not provide recommendations for the design of hardware.

ISO/TS 16071:2002 promotes increased usability of systems in combination with assistive technologies, when they are required. It does not cover the behaviour or requirements of assistive technologies themselves (including assistive software)."

NOTE: Is more needed here? We want to encourage people to order the ISO document, right? I don't want to violate the ISO copyright restrictions.

JAN SENT ME INPUT ON THIS-THANKS!

Action Item 2 (for ALL): W3C QA and WCAG References

Consult W3C QA WG page for overall information.

For quick references, consult:

• Checklist of Checkpoints for "QA Framework: Operational Guidelines
• Checklist of Checkpoints for "QA Framework: Specification Guidelines
• Checklist of Checkpoints for "QA Framework: Test Guidelines

Consult Development of WCAG2.0 Techniques for more information. There's been talk of a Gateway techniques document; HTML techniques seems most mature currently.

Action Item 3: Quality Assurance Issues (Including Test Suite Issues)

(NOTE: These are subject to change, but are meant as a starting point for discussion. Each of the items below has priority 1,2, and 3 subitems. The priority 1 subitems can serve as a starting point for discussion; priority 2 and 3 subitems are likely to be changed. Also see quick reference links in Action Item 4)

Operations Issues

• Commitment to Quality Assurance in Working Group activities.
• Commitment to resource level for Working Group QA activities.
• Synchronization of QA activities with the specification milestones.
• Definition of the QA process.
• Planning of test materials development.
• Planning of test materials publication.
• Planning of the transfer of test materials to W3C if needed.
• Plan for test materials maintenance.

Specifications Issues

• Identify what needs to conform and how.
• Address the use of profiles, modules and levels to divide the technology.
• Identify the relation between deprecated features and conformance.
• Define discretionary items.
• Allow extensions or not
• Clearly identify conformance requirements.
• Document the conformance policy.
• Specify how to make conformance claims.
• Provide test assertions

Test Issues

• Define Testing Strategy
• Identify testable assertions
• Support test material metadata
• Define the process for executing tests
• Document, package, and test the test materials before publication
• Define the mechanisms for reporting and publishing test results

Action Item 4: ATAG2.0 Test Suite Process Document

(NOTE: From template on QA site - cut and paste add "body" and "html" tags to display as standalone document)

ATAG2.0 Test Suite Process Document (DRAFT) - 15 Mar 04

Introduction

Requirements

• The ATAG2.0 TS should be developed in a public forum such that participation and contributions from parties interested in developing the AUWG TS is possible and welcome.
• A test material framework should be developed to provide a uniform method for producing and managing the ATAG2.0 TS
• The ATAG2.0 TS should be as comprehensive, functional and complete as possible
• A standard approach to ATAG2.0 TS IPR which meets the needs of both contributors and users should be established
• The ATAG2.0 TS should aid implementors in developing ATAG2.0 implementations. Validation and certification of these implementations are outside the scope of this work.
• The ATAG2.0 TS will be hosted on the W3C site once developed. It may also after development be mirrored at various sites in order to simplify access and enhance availability to the community.

Parties Involved

QA-related Communication

The public web page for QA information is named /Test/ and is available at: http://www.w3.org/AU/Test/. This page includes links to this process document, the ATAG2.0 TS, and information about the ATAG2.0 TS including:

1. an overview or introduction to the ATAG2.0 TS
2. releases
3. how to participate
4. how to use the ATAG2.0 TS
5. ATAG2.0 TS principles
6. acknowledgements

Development Framework

The ATAG2.0 TS will be produced in a public framework with contributions from members of the AUWG, W3C, or the community at large. The framework will provide a stable mechanism for submission of test materials, saving information about those tests, version handling, use of the tests, and so forth.

Contributions of test materials should be submitted to the ATAG2.0 test submission mailing list atag20ts-submit@w3.org. An archive is available at public-atag20-testsuite@w3.org?

The process for test materials contributions is:

• Contributions of test materials are submitted to the framework
• Receipt of the contribution is acknowledged. This is done via email directly to the submitter, with a copy sent to the mailing list. The test receives the "Received"; status.
• The contributed test materials together with the necessary documentation is saved for future reference.

• Relevance for the particular requirements of ATAG2.0. It can be decided here that the particular test is irrelevant to the ATAG2.0 TS, because it does not test a normative requirement of the specification.
• Part of the specification being tested
• Quality of code and documentation. The code must be stable and not contain any errors, and if written in an XML-based language, must be well-formed. Documentation must be written in accordance with prescribed information used to formulate and support the ATAG2.0 TS.
• Scenarios that underlay the particular test methodology and its intended scope.

Tests that are judged to be inappropriate for publication are returned to the contributor. Such tests are archived and are not included in the ATAG2.0 TS.

At any time, any member of the community or WG may dispute the validity of any test in the ATAG2.0 TS. This is done by raising the issue on the ATAG2.0 TS mailing list. If the dispute is felt to be valid by members of the WG, the status of the test is changed to "disputed", and the test is referred to the WG for detailed review. The possible outcomes of the review are:

1. error-free - the test and its accompanying documentation have undergone review by the WG and have been found to be free of errors and relevant to the specification
2. disputed: test error - a consensus exists in the WG that the test is in error. A resolution is formulated and the test is changed accordingly.
3. disputed: specification error - the consistency, clarity, etc. of the part(s) of the specification addressed by the test is questioned. The WG agrees that a problem exists with the specification and submits the test and the associated dispute to the WG's erratum process.

Each contribution to the ATAG2.0 TS must be fully documented. The documentation provides such information as:

1. name of submitter, date of creation, and, if appropriate, organization
2. suggested ID for each test
3. part(s) of the specification being tested. Areas in the specification being tested should be suitably indicated by either a link to the relevant part of the specification or a text annotation to the relevant part of the specification
4. the expected validity of the test
5. if appropriate, a textual annotation containing remarks on the test in human-readable format, for example, any scenario which prompted the writing of the test

Conformance

The ATAG2.0 TS aims to help implementors write applications that support ATAG2.0. In no way are these conformance tests to be construed as providing certification or branding of ATAG2.0 implementations. The only claim that could be made is that a particular implementation is conformant to a particular version of the ATAG2.0 TS. There are two cases of results of running the ATAG2.0 TS:

1. The implementation fails at least one aspect of the test materials associated with normative requirements of ATAG2.0. In this case, it can be asserted that the implementation fails to conform to ATAG2.0.
2. The implementation passes the test materials. In this case, all that can be asserted it that the implementation is conformant to that particular version of the ATAG2.0 TS.

Licenses

The Contributor hereby makes certain materials, as described below (the "Materials";), available for use in supporting the World Wide Web Consortium (W3C). The Contributor hereby grants to MIT, ERCIM, and Keio (the "W3C Hosts";) a perpertual, non-exclusive, royalty-free, world-wide right and license to use, copy, modify, create derivatives of and distribute the Materials, in whole or in part, solely in connection with the W3C, and to allow others to do the same. W3C Hosts will publish and distribute the Materials, or any modifications or derivations thereof, pursuant to the W3C Software License (20021231), and/or the W3C Document License (20021231) as modified from time to time.

The Contributor represents that she/he has all rights necessary to contribute the Materials, and that use of the Materials as comtemplated berein by W3C Hosts does not violate any copyright, patent, trademark, or contractual obligations.

The Contributor agrees that any derivatives of the Materials created in connection with the WS3C shall be owned by the W3C Hosts. Any publication or distribution of the Materials or any derivative thereof, will retain attribution of authorship to the Contributor. Whenever modifications are made to the Materials, this fact, and the nature of the modifications, will be clearly identified in the distributed version thereof. The W3C Hosts make no commitment to support or distribute the Materials.

The ATAG2.0 TS produced by the AUWG will be distributed under the W3C Software License.

Publication

The ATAG2.0 TS will be published on the W3C AUWG website, available from http://www.w3.org/AU/Test. The ATAG2.0 TS will be published after thorough review by the WG and the community, and with consensus of the WG. The ATAG2.0 TS will contain the conformance disclaimer as stated previously.

The AUWG encouragtes the publication of test results. To facilitate publication of results, the WG will make available on its webspace, a place to publish test results. Only test results that have been supplied or approved by the developer of the tested implementation will be considered for publication.

The WG will also consider a "members-only" webspace to publish developer-submitted test results. This restricted publication of results may be used for situations in which the specification and/or tests are being debugged (for example, when the specification is in CR and a set of tests are being used to assess the features of the specification by determining what is being implemented)

Test Material Maintenance

The AUWG is committed to maintaining the quality and relevance of the AUWG Test Suite. This will be done by ensuring that the test materials are consistent with any erratum and are updated as needed to reflect the current versions and status of the specification. The procedures dispute above will be used to accept new contributions as well as to handle tests that are in dispute.