Author: Dimitris Dimitriadis
Date: 20021002
Status: QA WG Disussion Draft (for 2002 Tokyo F2F)
This document revisits a thread from the QA WG mailing list earlier this year (spring 2002). In short, the thread discussed the necessity of forming a specifically devoted Test Group within the QA activity in order to deal with issues that will surely arise, once the Test Guidelines documentation is finished and released to the public. In what follows, I re-state the original content of that thread, extend it where necessary, and include arguments that were put forth.
Most of the QA-related test activities that have been produced up until this date in connection to W3C specs have either been internally produced within the particular specification's WG, or incorporated in coordination with external parties. This has lead to quality testing frameworks on the one hand, but in some cases very different and inconsistent, on the other.
According to the work that this WG produces, the W3C and its WGs shouls produce test suites that are checked against the same guidelines and checkpoints. This implies that they should share some basic functionality and design. In order to achieve this and ease the burden on the WGs that are to produce the Test Suites, I propose that the W3C form a special Test Group which has the characteristics below:
Simplify production of Test Suites (with the required minial set of quality requirements), enhance coordination between WGs and allow for easier asserting of cross-specification functionality. Also to increase the practice of the guidelines produced by the QA WG.
As elaborated below, it is my belief that the current QA WG will be busy explaining the Specification and Operations Guidelines and help Working Groups implement them. This is another reason why an organisatorially separate group should be formed to deal with particular test tasks.
The option of creating yet another Working Group, with a separate chair, the need for participation from the W3C membership may introduce too much overhead. If the Test Group activities could be dealt with in the current setup things would be fine. However, given the particular tasks needed in the TG, and looking at the profile of the current QA WG, I believe that we would, either way, need to extend our WG membership.
The Test Group should ideally be another (technically oriented) WG within the QA activity. The reason it should be a WG is that it needs a chair, for coordination, W3C staff allocated in order to make sure the technical architecture is there, as well as W3C member organizations and/or invited experts.
The Test Group should:
In my view, looking too much at details is only harmful. The criticism that has been put forth regarding the overhead introduced by creating another WG is serious, but needs to be separated into three, as I see it, issues:
All in all, with this document, I want to raise the issue of forming a TS focussed constellation failry soon. We need to gather existing results and provide guidance for future efforts in order to allow for greater simplicity and effectiveness in TS for all W3C specs. It is a continuation of the existing work, not separate from it.