W3C home > Mailing lists > Public > www-qa-wg@w3.org > January 2002

Re: Thursday telcon & agenda

From: Lofton Henderson <lofton@rockynet.com>
Date: Thu, 17 Jan 2002 09:08:54 -0700
Message-Id: <>
To: danield@w3.org
Cc: www-qa-wg@w3.org
At 04:47 PM 1/17/02 +0100, Daniel Dardailler wrote:

> > About #3. No new discussion drafts before telcon, but hopefully Friday.
>Should we talk about Ian's reorg of the TOC ?
>(not sure where it was sent)

Good idea.

Ian's message only had limited circulation amongst us (chairs and 
editors).  I have attached it below.  Please have a look.

For information:  the editors (dd, KG, LH) had a teleconference last week, 
and we endorsed the overall outline and many of the details of Ian's 
proposal.  Per the 1/3 telcon, email from LR, email from DD, we had been 
targeting a better implementation of Gd/Ck style.  Ian made a great 
contribution in giving us the outline that actually showed the suggestions 
in practice.

The next WG-only drafts (expected Friday) will largely reflect this.


>Sender: ian@w3.org
>Date: Thu, 10 Jan 2002 23:28:41 -0500
>From: "Ian B. Jacobs" <ij@w3.org>
>Organization: W3C
>X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.4.16 i686)
>X-Accept-Language: en
>To: danield@w3.org, karl@w3.org, lofton@rockynet.com
>Cc: ij@w3.org, w3t-archive@w3.org
>Subject: Comments on QA Framework: Processes and Operating Guidelines
>X-RCPT-TO: <lofton@rockynet.com>
>Lofton, Daniel, Karl,
>Below are some comments on the 2 Jan 2002 draft of "QA Framework:
>Process & Operational Guidelines":
>    http://www.w3.org/QA/WG/qaframe-ops.html
>I essentially ripped up and reorganized the document (a rough
>outline at least). Please don't take this the wrong way: I often
>do this as a way to get into a document and discover the issues
>the author had to wrestle with. I recently did this with the XML
>events draft as well. My comments may be a bit cryptic; don't
>hesitate to ask me to explain them. I hope some of the comments
>are useful, but feel free to ignore them. You can also forward
>them to the QAWG if you wish.
>The document adopts the guidelines/checkpoints model, which I'm
>happy to see. For each WAI document that adopted this model, the
>WG wrestled with the question of the depth of the table of
>contents. Each WG initially wanted a three-tiered table of
>contents, with guidelines grouping checkpoints and topic headers
>grouping related guidelines. In each case, the extra topic
>groupings were dropped to produce a shallower TOC.
>In other words, the TOC went from:
>    Topic 1
>      G1
>        checkpoint 1, checkpoint 2,
>      G2 ...
>      G3 ...
>    Topic 2
>      G4 ...
>      G5 ...
>    ...
>To the shallower:
>    G1
>      checkpoint 1, checkpoint 2,
>    G2
>    G3
>    ...
>The primary reasons for this were:
>    * The guidelines seem to suffice as an organization layer.
>    * People can't remember the extra layer of organization
>      anyway.
>    * It feels "cleaner" to have a straight list of guidelines,
>      numbered 1 - N.
>I thought of this as soon as I saw section (four levels
>deep) in the P&O Guidelines. I urge you to simplify the table of
>contents by removing some grouping layers. I suspect there will
>be resistance to this at first (as there was in each WAI WG), but
>my experience is that shallower TOCs are more effective.
>So, below I:
>  * suggest a simpler TOC
>  * suggest that some checkpoints are unnecessary
>  * reorganize the checkpoints
>  * use the WAI style of the imperative voice for
>    guidelines and checkpoints.
>Sample TOC
>    1. Introduction
>    2. Guidelines
>       1. In charters, describe QA goals, deliverables, resources
>       2. Ensure that policies for QA work are clear.
>       3. Promote clear communication with other groups
>       4. Make test suites/tools part of specification development
>       5. Plan for test suite maintenance
>    3. Working with the QA Activity [Essentially the current 2.6]
>    4. Conformance
>    5. References
>    6. Acknowledgments
>       [I moved Acks after References based on IETF RFC model
>        where the document progresses towards the most informative
>       (i.e., non-normative) information. This is a new practice
>       for me.]
>Note: Most of the rest of the prose in the document can be
>inserted as notes after checkpoints or guideline introduction
>material, or even subsections of "1. Introduction".
>It is my opinion that each checkpoint should be addressed to a
>consistent audience within each Guideline.  For instance, the
>checkpoints in the first Guideline below (on charters) should
>only be addressed to charter creators. Below I use the imperative
>as in the WAI guidelines, but I have no problem with making the
>subject explicit, as in "The charter shall...".
>  ==============================================
>  1. In charters, describe QA goals, deliverables, resources
>  ==============================================
>     1.1 Include a section on test suite/tool plans in the
>         charter. The plans should be at least a commitment
>         to "QA level four" (see range 1-11).
>         Note: I don't think it's necessary to have a checkpoint
>         that existing charters should be updated. That requirement
>         only applies once, when the QA documents are approved
>         within W3C.
>     1.2 Ensure that each charter makes explicit resource
>         commitments to QA (e.g., for staffing of test
>         suite development and maintenance.
>         Note: It's not necessary to say too much about
>         documenting resource commitments in the charter;
>         that's already required by the Process Document.
>         But it makes sense to say "Don't forget QA."
>         Note: There were several staffing checkpoints in
>         the document. In my opinion, one checkpoint suffices,
>         even if the resources are for different types of
>         work. Just enumerate them, if necessary, in the
>         checkpoint.
>     1.3 Identify where and how conformance materials will
>         be produced.
>         Note: The checkpoint about w3c providing a secure
>               and reliable repository can be deleted.
>  ==============================================
>  2. Ensure that policies for QA work are clear.
>  ==============================================
>     2.1 Ensure that W3C has ultimate responsibility for
>         evolution of test suites and tools.
>     2.2 Ensure a clear policy for branding materials
>         (e.g., conformance logos).
>  ==============================================
>  3. Promote clear communication with other groups
>  ==============================================
>     3.1 Identify the communication channel for appeals
>         of test validity.
>  ==============================================
>  4. Make test suites/tools part of specification development
>  ==============================================
>     4.1 Review technical guidelines before starting QA work
>     4.2 Synchronize specification writing and and development
>         of test materials early in the WD process. Keep
>         them in sync.
>     4.3 Publish a QA process document that explains:
>          a) communication mechanisms with the test suite managers,
>          b) contribution mechanisms,
>          c) review procedures for contributions,
>          d) disclaimers regarding certification
>         Note: Provide boilerplate text (informal) for the
>         disclaimer: passing all of the tests does not guarantee
>         full compliance of implementation to spec, and failing
>         the test suites means failing specific tests for specific
>         targeted features.
>         Note: I don't think it's necessary to have a checkpoint
>         requiring the process document to point to the test
>         materials; this seems like an ordinary thing one would
>         do.
>         Note: I don't think it's necessary to have a checkpoint
>         on usability of test materials for two reasons:
>         (a) people will make use of the most usable instance
>             available.
>         (b) unless you explain what makes the test suite
>             usable, the checkpoint won't be helpful. If you
>             have points for making a test suite usable,
>             add a guideline on test suite usability and
>             include specific checkpoints under it.
>  ==============================================
>  5. Plan for test suite maintenance
>  ==============================================
>     5.1 Designate a task force to maintain the test suite
>         and maintenance procedures.
>     5.2 Support specification versioning and errata.
>     5.3 Ensure that test materials are documented,
>         self-contained, and reusable by other parties.
>     5.4 If the WG is to close, ensure that test suite
>         maintenance is handed off to another entity.
>Other comments
>  1) In section 2.2.1, the range of possible commitments would
>     be easier for me to compare if each additional point added
>     just one piece to the previous point. There seem to be
>     several axes:
>       - WG composition
>       - What's in the spec
>       - Extent of test materials
>     I could see organizing the levels like this:
>         WG composition | In spec | Extent of materials
>     -------------------------------------------------------
>     1:  Some for QA    |         |
>     2:                 | Example |
>     3:  Some for QA    | Example |
>     4: ....
>     In other words, it would be great if the levels built on each
>     other in a predictable and incremental way. The order in the
>     current rating 1 - 11 is hard to follow.
>     Note: I think you can delete point 1 or rephrase it
>     since "promises to think about" is not verifiable.
>  2) Point 11 in the table includes the phrase "complete test
>     suite." It occurred to me that the I don't know what a
>     complete test suite is. It would be useful in the Intro
>     to make very clear:
>     a) Objects. What is a test suite? Catalog? Etc. Perhaps these
>        items will be discussed in the glossary, but it will be
>        important to know what the deliverables are.
>     b) Processes. The reader should have a better understanding
>        of the processes associated with the objects (and the roles
>        of people in these processes as well, e.g., "contributor",
>        "moderator"). This will help people understand the
>        associated requirements.
>        I'm thinking of the diagram for the XSLT test suite, that
>        shows where information comes from, who manages it, when
>        approval takes place, etc. Part of this might belong in
>        another document more about test suites and less about WG
>        processes. But sections and are about
>        contribution and review processes.
>        In short, I think the reader will have an easier time
>        if the Intro explains in broad strokes how the pieces
>        fit together.
>Ian Jacobs (ij@w3.org)   http://www.w3.org/People/Jacobs
>Tel:                     +1 718 260-9447
Received on Thursday, 17 January 2002 11:08:39 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:25 UTC