Editors: Dimitris Dimitriadis (DD), lofton Henderson (LH), Kirill Gavrylyuk (KG)
This version submitted by DD
Introduction
The anatomy of this particular document is the following
- Proposed partition of the W3C QA Document family
- Examples in connection to each header
- Explanatory remarks with a short recap of the discussion between DD, LH and KG
Document Family Partition
In an off-list corresepondance, the partition belwo was originally introduced by LH:
- Framework: Introduction
- Framework: Procs&Ops Guidelines
- Procs&Ops Examples and Techniques
- Framework: Spec Guidelines
- Spec Guidelines Examples and Techniques
- Framework: Technical Guidelines (Proposed wording: Test Suites and Validators -- Methods and Designs)
- Technical Guidelines Examples and Techniques (Proposed wording as above)
DD, LH, and KG feel that the documentation partition serves our purposes well. Below you will find explanatory remarks on each section and its content.
Introduction
This document will consist of the following
- Description of the W3C QA Activity (IG/WG)
- Short review of the charters
- Description of interedependencies with other W3C WGs as well as pointers to other charters and process documents, where appropriate
- Short description of and pointer to other documents in the W3C QA realm
Process and Operations
This document deals with the workings of the W3C QA IG/WG in more detail. Especially, it stresses the communication channels available, pointers to minutes of meetings and telcons, as well as interdependencies and relations with other IG/WG. This is especially valuable since ther Proc&Ops will be consulted by new groups forming while writing their charter.
Also, we may want to include a section on QA reps in other IG/WG. We should give an account of how the QA activity is represented in other bodies of the W3C (this is an issue that needs clarification with Domain Leads and possibly the chair of the W3C).
Process and Operations examples and techniques
Pointers to existing documentation, tutorials, matrices and relevant cooperation with other IG/WG.
Specification Guidelines
This is possibly one of the most important documents, especially since it deals with issues relating to automatic generation of tests and quality assurance assertions (applicable only to fairly recent technology specifications perhaps, older specs will have to be dealt with in a more manual fashion). Here we give an account of guidelines directed towards WGs, since they are the ones writing specifications (the guidelines should be of interest to the W3C at large, of course). In particular, we stress the relevance of writing specifications in a fairly verbose and detalied way, in XML, allowing for the following:
- Automated processing of specifications, making possible
- Automated generation of tests
- Automated generation of test assertions (however, modelling assertions is not a bit easier than properly modelling semantivs to begin with)
- Generation of matrices (on the particular technologies)
- Easier to assign tasks and issues to specification production as well as easier integration of testing with the specification production process.
- For technical specification, as eg. DOM, automated generation of smoketests of entire specs, allowing for forming a small set of basic interface tests that both writers and implementors of specifications can use in parallel to producing specifications.
- Testing of interdependencies of specifications and technologies.
Some of the issues here are
- Not all technologies gain from having too granular a grammar on which the specifications are based (defining mouse events is a different issue than defininf the XML processing model, for example)
- Older specifications will not gain from this general framework, however, the editors propose that this be pursued from now onwards, dealing with older specifications manually.
- Having a granular language for describing the technology specified introduces the question of what it is that is being tested; if youhave a granular enough specification ML, you can easily write tests that expose weaknesses of the spec itself. This is an altogether different issue than writing tests that fail fpr particular implementations. If pursued, this section needs to safeguard against writing unalterable sepcifications (this is also one example of why specification writing and testing is two different perspectives of the same thing [DD].
- Since we cannot write a detailed anough ML, we may want to introduce a superset of such a language instead of going all the way and limiting people to use a particular tagset. Again, not all technologies aim at solving the same problems.
Specification guidelines, examples and techniques
Pointers to existing frameworks (DOM; SVG etc.)
Framework: Technical Guidelines (Proposed wording: Test Suites and Validators -- Methods and Designs)
As indicated above, the proposed wording aims at clarifying the distinction between the methodology of QA at large, of which testing, specification etc. are parts, and the technical implementation itself. This document will deal with Tests and Test Suites. It should mainly consist of
- How to write tests given a particular specification (and possibly guidelines for producing testing documentation)
- Architecture for testing (what software could be used, what platforms could serve as servers if testing is done over the web etc. (this, however, touches on a series of issues among which licensing is the most difficult one, and it is not clear where relevant decision are to be taken).
- How to extrapolate relevant assertions from specifications, given that they are granular enough (if not, one can simply provide test assertions or semantic requirements in prose, as in the DOM TS).
Technical Guidelines Examples and Techniques (Proposed wording as above)
Again, pointers to relevant projects and test frameworks.
Known issues
- In our discussion, a few things that need clarification have already surfaced. The most important is terminology. For our own and our readers and implementors sake, we need to come up with a worked-through terminology. The best example of confusion that may arise is that the concept "framework" gives rise to a series of problems; is it to be understood as everything a particular (in this case W3C QA) line of methods and design patterns gives rise to, or just the technical framework on which this is implemented?
- interWG communications: how much of the above should normatively be implemented by the various WGs?