Re: new draft "Framework: Process & Operational Guidelines"

I'm fine with leaving it as is and mentioning what you just said in
the intro of the section

> Thanks, Daniel!
> Quick comment on the last question:
> >> 3. WG relationship to QA Activity
> >Is time the only reason why this section is not turned into a
> guideline/checkpoint layout ?
> >Should be mentioned if that's the case and we don't have time to change
> it before publication. I think we should take the time though.
> 
> No, it was intentional. This chapter doesn't require anyone outside of
> QA WG to conform to any rules, it rather describes the process QA WG
> follows. I found it a bit confusing to write guidelines/checkpoints for
> ourselves, hence left this chapter as plain text.
> Any arguments why should we change it, except following WAI example?
> 
> 
> > -----Original Message-----
> > From: Daniel Dardailler [mailto:danield@w3.org] 
> > Sent: Friday, January 25, 2002 6:42 AM
> > To: Lofton Henderson
> > Cc: www-qa-wg@w3.org; ij@w3.org
> > Subject: Re: new draft "Framework: Process & Operational Guidelines" 
> > 
> > 
> > 
> > 
> > >    This document is a Working Draft, made available by the [18]W3C
> > >    Quality Assurance (QA) Activity for discussion on the QA email 
> > > list.
> > 
> > a bit picky but it's "for discussion on the QAWG mailing list."
> > 
> > >    All review comments against the previous version
> > >    have been accepted and implemented, except as indicated in an
> > >    associated [19]disposition of comments document. This 
> > version still
> > 
> > The disposition of comments actually lists a bunch of "taken" 
> > comments, so it's unclear if it is not up-to-date or if the 
> > description here is wrong.
> > 
> > 
> > >    This document is part of a family of QA Framework 
> > documents designed
> > >    to improve the quality of W3C specifications as well as their
> > >    implementation by solidifying and extending current 
> > quality practices
> > >    within the W3C. The QA Framework documents are:
> > >      * Introduction
> > >      * Specification Guidelines
> > >      * Process and Operational Guidelines (this document)
> > >      * Technical Guidelines
> > 
> > I suggest we use the same order as usual
> >  
> > >    Not only
> > >    are the WGs the consumer of these guidelines, they are also key
> > 
> > consumers
> > 
> > >    Although not explicitly stated, the W3C Process Document 
> > supports the
> > >    development of conformance test materials.
> > >     1. "[.] groups may produce technical reports, review the work of
> > >        other groups, develop sample code or test suites, 
> > etc." (sec. 3.)
> > >     2. "W3C should make every effort to maintain its Recommendations
> > >        (e.g., by tracking errata, providing test bed applications,
> > >        helping to create test suites, etc.)" (sec4.)
> > >     3. In an effort to meet these suggestions and address the
> > >        implementation requirements of the Process Document, 
> > some Working
> > >        Groups have included the development of conformance 
> > materials as
> > >        part of their CR-exit and PR-entrance criteria 
> > (e.g., SVG, UAAG,
> > >        XSL-FO). This makes sense, since it is natural for 
> > test suites and
> > >        implementations to develop in parallel - each is a 
> > help to the
> > >        development of the other.
> > 
> > I'd put some font emphasis on the "test" related words of the 
> > above extracts, just to be clearer.
> > 
> > >   1.2 Navigating through this document.
> > 
> > I think this section should acknowledge the reuse of the WAI 
> > model, so 
> > that people can actually switch their mind to it, instead of 
> > wondering 
> > what's different but similar.
> > 
> > >   1.3 Terminology
> > > 
> > >    The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", 
> > "SHALL NOT",
> > >    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 
> > "OPTIONAL" will be
> > >    used as defined in RFC 2119 [39][RFC2119].
> > 
> > Since the checkpoints define the requirement/normativeness 
> > level, and they use priority instead of verbiage, one may 
> > wonder why we say we're 
> > going to use the RFC2119 model. The WAI guidelines do not 
> > refer to it I think.
> > 
> > >    Checkpoint 1.1.Priority 1.In charters include section for test
> > >    suites/tools plans. Plan to have at least a commitment 
> > to "QA level
> > >    three".
> > 
> > The WAI style (sorry to always bring it up, but that's the 
> > reference in the area) is to put the Priority level after the 
> > checkpoint text, either in parenth or bracket, usually using 
> > a different color scheme. See /tr/wcag or /tr/uuag. I like 
> > the UAAG approach of just the letter in red and in parenth at 
> > the end of the line: (P1)
> > 
> > >    This 10-point enumeration is derived from a [52]proposal 
> > to the QA
> > >    mail list, after the 4/2001 QA Workshop.
> > 
> > First, it's unclear what relation exists between this 
> > commitment scale 
> > and the requirement expressed in the surrounding checkpoint, 
> > so it's kind of confusing.
> > 
> > Second, somewhow, I think the table is too complex and should 
> > be reorganized more efficiently: half of text is just a 
> > duplicate of the same sentence. David's original linear list 
> > was simpler if I remember correctly.
> >  
> > >    Checkpoint 1.2.Priority 2.In charter, specify completion and
> > >    publication of test materials to be a criterion for CR-exit and
> > >    PR-entrance.
> > 
> > >    Checkpoint 1.3.Priority 1.In charter, address QA staffing 
> > > commitments.
> > 
> > I rather reverse the priority of those two: what's more 
> > important is that the WG promises to complete its QA work, 
> > not that they identify QA staff ahead of time.
> >  
> > 
> > >    The last item requires some elaboration. Depending on 
> > the nature of
> > >    the test, it may actually be sufficient to judge 
> > non-compliance of the
> > >    subject. Based on [54]taxonomy the following is valid:
> > >      * for conformance testing of the (web) content: statement 1 is
> > >        false, but statement 2 is true
> > >      * for content/driven conformance testing of the 
> > tools/product: both
> > >        statements are true
> > >      * for interfaces (API) conformance testing of the tools or
> > >        libraries: both statements are true
> > > 
> > >      [54] http://www.w3.org/QA/WG/2002/framework-20020118/TAXONOMY
> > 
> > broken link
> > (should run ,checklink on all documents, Dom will not publish
> > otherwise)
> >  
> > >    Guideline 4. Allocate staff for QA process.
> > > 
> > >    Before starting QA development, Working Group has to 
> > appoint several
> > >    key participants of QA process.
> > 
> > not sure a spell checker is going to catch that, but the 
> > english is poor, I suggest:
> >   Before starting on QA development, a Working Group has to 
> > appoint several
> >   key participants for its QA process.
> >  
> > or some such
> > 
> > >    Depending on the plan for origin of the test suite WG 
> > can appoint test
> > >    moderator from
> > 
> > same here, missing articles..
> > 
> > >      * WG members (and other W3C members)
> > >      * W3C QA staff
> > 
> > in fact, it can come from anywhere, it just has to be part of the WG.
> >  
> > >    When test suite is developed outside of Working Group, a 
> > moderator
> > >    usually already exists where test suite coming from. It 
> > is recommended
> > >    to invite moderator to become WG member, since he/she 
> > needs access to
> > >    all WG materials and resources. See also section on 
> > [55]transferring
> > >    test suite from outside W3C.
> > 
> > (english getting really hard to read...)
> > 
> > >    Guideline 6. Produce QA process document.
> > 
> > long guideline, either we'll have to split it or to combine 
> > checkpoints (by experience). not needed for first PD.
> >  
> > >    Checkpoint 6.1.Priority 2.Review Technical Guidelines 
> > before writing
> > >    QA Process document and starting test materials development.
> > 
> > not verifiable really, so should not be there
> >  
> > >    Taking into account the nature of the future standard, 
> > before starting
> > >    test suite development, test suite implementer (WG or 3rd party)
> > >    should look at [57]Technical Guidelines and decide what 
> > approach to
> > >    take for test materials organization, tests criteria, etc.
> > > 
> > >      [57] 
> > > http://www.w3.org/QA/WG/2002/framework-20020118/QA-TECH-GUIDE
> > 
> > broken link, maybe should point at DOM process for now
> >  
> > >    Checkpoint 6.2.Priority 2.In QA Process document, 
> > explain contribution
> > >    process.
> > 
> > this one looks like a dup of 6.5, or 6.5 + 6.2, in any case, 
> > it looks like it's already covered by following checkpoints
> >  
> > >    The following example may serve as a guideline for tests review
> > >    process
> > 
> > maybe mention that this kind of materials will eventually go 
> > into the example/technical companion
> > 
> > >    Checkpoint 6.7.Priority 3.In QA Process document, 
> > describe how vendors
> > >    can publish test results for their products, if applicable.
> > 
> > this one similar to 3.2 and 3.3, should probably go there or disappear
> >  
> > >    The following procedure is recommended:
> > > 
> > >    Before transfer WG with the help of QA WG:
> > 
> > same remark as above, example/technique stuff, ok to keep in 
> > FPWG but should mention that it will go away eventually
> >  
> > > 3. WG relationship to QA Activity
> > 
> > Is time the only reason why this section is not turned into a 
> > guideline/checkpoint layout ?
> > 
> > Should be mentioned if that's the case and we don't have time 
> > to change it before publication. I think we should take the 
> > time though.
> > 
> > >      * The conformance level satisfied: "Level 1", "Level 
> > 2", or "Level
> > >        3".
> > >    Example:
> > > 
> > >    This QA processes and operations of this Working Group 
> > [???] conform
> > >    to W3C's "QA Framework: Process & Operational 
> > Guidelines", available
> > >    at [...tbd...], Level 2.
> > > 
> > 
> > you mean A, AA, etc
> >  
> > 
> >

Received on Friday, 25 January 2002 13:29:21 UTC