This is a draft process document. It is based on the detailed outline by Lynne Rosenthal (http://www.w3.org/QA/WG/2002/11/ProcessDoc-20021121.html) as well as ideas presented in the first OpsGL draft, Karl’s earlier draft (http://lists.w3.org/Archives/Public/www-qa-wg/2002Jun/0078.html), as well as things that have been mentioned in email and at meetings.
This Process Document is broken into 3 major sections.
@@@ Note: Hopefuly there is more order than before, it's a work in progress.
@@@ Preface each section with a link to the relivent information in the W3C Process guidelines. Include brief summery of 'default' state with some indication of why the current state needed expansion, refinement, or clarification.
The W3C process document There are 2 types of invited experts: those with full membership and those that are invited to participate on a specific topic.
@@@ w3c process document on WG participation - Section 4.2.3, invited experts are explicitly mentioned and there is a process to follow to be able to participate in w3c wg's.
@@@ particpation commitment for invited experts is also menionted in the charter, section 11. It references the w3c process document for requirements of meeting attendance and participation.
Any WG member can request that an expert be invited to participate in the WG. If a Non-W3C members wishes to participate in the WG they may petition for membership by contactking the WG chair.
To be an invited expert they must meet the requirements for participation set in the W3C process document (@@@ link to sec 4.2.3 in process doc). All invited experts must provide their credentials and state their ability to meet the WG commitment as defined in section 11 of the charter(attend meetings, 4-8 Hours/ week) (@@@link to charter)
Acceptance of an invited expert is decided by a vote of the WG.(@@@ or chair, this needs to be decided by vote...)
Because of the depth and bredth of topics that the QAWG cover, potentialy touching a number of other Working Groups or processes, there may arrise topics of discussion that are beyound the expertese of the WG members. In such a case it may be adventagious for the WG to invite topical experts to provide more information or better insight into such topics.
An expert may be invited to participate in a specific discussion or meeting of the WG if the expert has particiular insight or knowledge in the topic being discussed.
Any WG member may request that an expert be invited for a meeting or discussion. When a request is made the WG member making the request should present the credentials of the expert. Once an invitation to an expert has been proposed the WG must vote to approve the invitation. If the invitation is approved, the WG chair extends the invitation on behalf of the WG. This invitation should also be sent to the WG mail list.
As opposed to invited experts, there are no membership commitment requirements for a topic expert. The participation of a topic expert is limited to a single instance. They do not need to follow the guidelines for participation as set forth in section 4.2.3 of the W3C Process Document or section 11 of the charter.
@@@ Do the limitations in the W3C process document prohibit us from doing this? Topic Experts are not covered in the process document at all, how ever it does say "The request to join must indicate that the nominee wishes to particpate as an invited expert (even for participation in a meeting on a one time basis)." Section 4.2.3
When meetings are held, face to face or distributed, there arrise a number of occasions when formal decisions are made. Formal decisions include such things as votes, resolution of issues, changes in process or operations, and invited experts. The W3C Process Document describes the voting process. (@@@Section 4.1.2).
The WG has a quorum requirement for formal decissions to be made. Quorum is definined as having at least 1/3 of the WG members present. Decisions made when quorum is not present are not binding or valid.
WG members not present during qurum may raise formal objectsions to decisions made. Any dissent to a decision made in a meeting with quorum is handled as outlined in the W3C Process Document.
- When an action item or assignment results in an unexpected change or modification to fulfill the assignment, it must be called to the attention of the WG.
- Announce and if necessary, present an overview of the change at a telecon. Not only does this ensure that the WG is aware, but it is also documented in the minutes.
- Examples: Reformatting the OpsGL to look like the SpecGL. Writing an issue resolution from a general agreement of what needed to be done.
- When does something get publicized to the IG
- All deliverables get announced to the IG. When?
Upon completion, email to AI editor with AI #
- Issues pertain to the Framework Guidelines (intro, ops, spec, test) and their companion ExTech documents, process, and any other WG deliverables
- @@@ Do all comments on a document become issues? - Issues are derived from comments received in email and/or from meeting discussions.
- All comments on documents must be acknowledged, reviewed and if warranted, made into an issue, resolved, and commenter informed. This includes comments from within the WG, IG, and public.
-Originator
– either WG member or external party Sends in a comment or inquiry that results in an issue
-Chair (document editor?)
– receiver of the issue. Starts the process, puts issue discussion on agenda, and assigns ownership
- Owner
– member of the WG who is responsible for the issue (may not be the originator) - Acts as liaison, if submitted by external party. If needed, formats the issue, presents it for discussion, contacts the originator regarding disposition
- Issues Editor
– owns the Issues List
- The telecon following the receipt of a comment or inquiry is the start of the issue tracking. The chair (or document editor) initiates the issue processing.
- The WG determines if this really is an issue, and if so, the Issues Editor enters it into the Issues List. If the issue isn’t in the proper format, the task of formatting the issue can be delegated.
- Each issue is assigned an owner, who is responsible for ensuring the issue is processed. If the comment is made from a member of the WG, then that member becomes the owner, else the chair or editor assigns an owner. For issues received from non-WG members, the owner acts as liaison between the external party and the WG. The owner sends an acknowledgement to the external party regarding receipt of the comment and provides the issue number and url of the issues list if appropriate, and upon resolution, informs the external party.
- Issues are categorized and prioritized. The chair or document editor decides the priority and requests time on a meeting agenda for discussion. Issues can be discussed via email but resolved at meetings (telcon or f2f). It may be necessary to continue discussions beyond a single meeting. It may be necessary to assign an action item to a WG member to get more information and/or discuss the issue with the Team contact.
- When there is consensus on the outcome of the issue, a resolution is written and recorded in the Issues List. The Issues Editor can either write the resolution or delegate it. The issue is marked as closed.
- The issue owner informs the originator that the issue has been resolved. This is an important step, especially when the originator is not a WG member.
- The WG is informed of updates to the Issues List by either announcing this at a meeting so it appears in the minutes or sending email to the QAWG mailing list. Periodically, email is sent to the IG list to inform them of updates to the Issues List
-Once an issue is closed, the chair or editor rules on requests to reopen or continue arguments on closed issues (remember issues can only be closed if there was a quorum present when closed). There must be new evidence uncovered or altered situation in order to reopen
Potential relationships with other W3C WGs include: (taken from the OpsGL)
--- Liaison and consultation,
--- reviews,
--- adjudicate entry and exit criteria,
--- QA resources supplement,
--- Resolution of external QA requests
-There are different types of communiquÈs - some more formal than others. Communication with QAWG may be received as email to the chair, email to the WG mail list, IG mail list, or from a verbal request.
- All questions or inquiries must be answered.
- Any WG member can answer, however chair or chair’s designee is responsible for making sure the WG responds with some type of acknowledgement or answer.
An important function of QAWG is to review W3C work with QA issues in mind. This is done as early as possible, at the charter stage, during the document process, and during the development of conformance materials. The QAWG when resources are available will review and provide guidance to WGs.
- Requests can be handled on a case by case basis – with the chair bringing the request to the attention of the QAWG. Alternatively, the QAWG can appoint a team to handle all incoming requests.
- All requests are put on the agenda for the next meeting, where they are introduced and preliminary action is taken.
- Chair solicits interest. A subgroup is formed and a leader is appointed. A subgroup can consist of 1 person. If there are no volunteers, the chair can appoint someone.
- If there is no one available that can do the review in the allotted time frame and/or this is outside the expertise of those who are available, the chair responds that the QAWG doesn’t have the resources or expertise to perform the review.
- If a subgroup is formed, the group makes comments. These comments are submitted to the QAWG for review and approval. When approved, the comments are sent as QAWG comments. The QAWG can waive review and allow the subgroup to send the comments directly as QAWG comments. If time doesn’t permit a QAWG review, then the comments can be sent as an individual comment.
- Review of technical documents and test materials includes verifying that the checkpoints of the appropriate QA Framework documents are met and providing guidance to the requestors on how to meet the checkpoints.
- QAWG is a source of information – people come to for guidance and tools.
- May get requests to help build conformance materials, including drafting documents or conformance sections and developing test suites.
- Upon receipt of request, chair evaluates the request, estimating the type of resources and expertise needed, and duration of effort. Chair can appoint someone to do this. Present to QAWG, who then decides if we can do it. If yes, a person or subgroup is formed to assist. If no, the chair informs the requester.
- QAWG acts as liaison and facilitator between external organization and appropriate W3C WG. Puts the right people in contact with each other.
- May need to ‘translate’ conformance terminology and technology for the 2 groups.
- Provides lessons learned and examples from other groups that have already been through this.
- Asked to participate in or present QA to other groups and conferences. Goal = education and outreach by
-Collaborate with other groups
-present work at conferences -participate in other groups (WG, IG, etc) meetings, including workshops to establish a group.
[ed note: The following applies to the development, review, etc. of test materials corresponding to the QA Framework documents and any other deliverables with conformance implications.]
The WG must appoint a Moderator. The Moderator must be voted on by the WG.
WG Test Materials should have it's own web page. This web page should be linked to from the WG home page. The Test Materials web page should be used for posting announcements, news, deliverables, releases, information, FAQs and information on how to participate.
Discussions on Test Materials should use the existing QA WG mailing list (www-qa-wg@w3.org). Once WG consensus is reached on issues concerning Test Materials, thise decisions should be published to the QA IG mailing list (www-qa@w3.org).
- the following applies to contributions made by QAWG members, other W3C members and WGs, as well as organizations and individuals external to W3C
- test materials, either whole or in part (e.g., test suites or individual tests) submitted via the mailing lists
- acknowledgement by the moderator, that has been received and in ‘received’ status
- IPR or license applied to submissions?
- cursory review to see that all required info and format properly; if no, send back
- at least 2 reviewers?
- review for relevance, accuracy, scope, clarity, aspect of spec begin tested, quality of code and documentation
- run each test on different implementations – shows device independence
- group reaches consensus on accept, reject, or defer decision while issue forwarded to WG for clarification
- a test can be rejected even if all the review criteria met and reviewers think it’s a good test, but there is a discrepancy with the interpretation of the Spec.
- if test accepted and decided to be stable, which means that it is relevant, that it indeed tests a particular identifiable part of the spec and that it is not wrongly written, it becomes ‘reviewed and stable’
- if test is accepted but needs clarification, mark status as ‘reviewed and pending’
- tests that at any point in the review are judged not appropriate, receive ‘inappropriate’ status. And are archived
- when ‘reviewed and stable’ tests found to be erroneous (i.e., not stable), then go back to Review step (13.2)
-each test material comes fully documented with regard to the following:
-- name of submitter, date of creations, and if appropriate company
-- suggested ID for the test
-- Part of spec being tested
-- (if appropriate) scenario which has preceded the writing of the test
- completed test materials will reside in xxx space - developmental work and published versions via CVS?
-plausible to assume that changes and revisions will need to be made – which will be noted by different version of the test suite
- Moderator responsible for versioning and proper inclusion of tests in the various versions
- any comments received on published test materials follows Issues Processing, (step 5)
- use the Document License
- no branding
- Test materials aim to help implementers write specifications and conform to the guidelines. The test materials do not provide certification. The only claim that can be made is that a particular implementation is conformant to a particular version of the test material.