W3C home > Mailing lists > Public > www-qa-wg@w3.org > April 2002

RE: Ops Guidelines Issue

From: Lofton Henderson <lofton@rockynet.com>
Date: Mon, 22 Apr 2002 12:16:54 -0600
Message-Id: <5.1.0.14.2.20020422114554.02f3ab90@rockynet.com>
To: lynne rosenthal <lynne.rosenthal@nist.gov>, "Kirill Gavrylyuk" <kirillg@microsoft.com>
Cc: <www-qa-wg@w3.org>
At 07:15 AM 4/22/02 -0400, lynne rosenthal wrote:
>One way to do  this is to take out the "in the Charter" from the 
>checkpoints - and at the end of that Guideline, the last 2 checkpoints 
>are:  For new or rechartered WGs, put all this stuff in the Charter.  For 
>existing WGs, make sure that it is documented in the minutes and/or via a 
>separate WG document (e.g., Process document)

I'm not sure that it is a good idea to add conditionally applicable 
checkpoints, if they can be avoided.  Your first checkpoint is "N/A" for 
old WGs, and your second one is N/A for new WGs.

There seems to be some resistance to putting the conditionality into the 
Examples & Techniques, and I'm not sure why.  It would seem, if you look 
carefully at the WAI standards, that Examples and Techniques is exactly the 
place for this.  In WAI standards, "Techniques" tells you how to satisfy 
the requirements of the checkpoints in "Guidelines".

By analogy, looking at checkpoint 2.1:
===
In Ops Guidelines:
------
Checkpoint 2.1.  Address where and how conformance test materials will be 
produced. [Priority 1]

(Some descriptive prose, including, "The way in which the checkpoint is 
satisfied is different for new WGs versus existing WGs."

@@See Example&Techniques for Checkpoint 2.1@@

In Ops Examples & Techniques
-----
Checkpoint 2.1.  Address where and how conformance test materials will be 
produced. [Priority 1]

(...text for Examples...)

To satisfy this checkpoint, a new Working Groups must document "where and 
how" in the Charter.  (...more discussion...)

For existing Working Groups any one of the following will serve to satisfy 
this checkpoint:  1.) discuss in face to face meeting and minute the 
result; 2.) or separate WG document 3.) or ...etc.   (...more discussion...)
===

This aligns with the sort of usage in Extech that Mark commented on in the 
last telcon -- it is the sort of "how to satisfy" prescription that he was 
concerned was missing from our initial Extech draft (i.e., we seemed to be 
aiming for all examples, and no techniques).  That is the only awkwardness 
I see with having this in the next Extech version -- most of the Extech 
content will be descriptive examples, and there will be relatively little 
such prescriptive techniques.

-Lofton.



>At 06:22 PM 4/21/2002, Lofton Henderson wrote:
>>At 04:45 PM 4/18/2002 -0700, Kirill Gavrylyuk wrote:
>>>Thanks, Lofton,
>>>
>>> >do we intend to prescribe for existing WGs?
>>>I agree that we need to give a roadmap of what to do for existing WGs.
>>
>>Yes, but I think that the tricky or sensitive point might be:  how 
>>forcefully should the Framework present the Gd/Ck for existing 
>>group?.  I'm thinking specifically about Ops guidelines 1 & 2, about 
>>commitment and resources.
>>
>>> >The only difference is *how*, i.e., the technique.
>>>Ok, I see your point. But for the existing groups it is hard to verify
>>>if the checkpoints, since the "somehow" isn't deterministic and depends
>>>on the circumstances...
>>
>>Actually, I think that we could state something deterministic for 
>>existing groups.  I was using "somehow" as a placeholder.  For example, 
>>an existing group could satisfy the Ops checkpoint 1.1 by discussing QA 
>>commitment in teleconference and documenting the resolutions in archived 
>>minutes.
>>
>>The only tricky part of this is:  should these checkpoints hold existing 
>>groups to the same level of commitment as new groups?  E.g., should 
>>achieving priority 1 (Level A) conformance for existing group require 
>>level-3 commitment, documented in minutes?
>>
>>I'm unsure of this detail now.
>>
>>>May be just having a section in examples document explaining how to
>>>apply the checkpoints to existing WGs would suffice?
>>
>>That is what I had in mind, except I was also proposing to drop from the 
>>statements of the checkpoints the phrase, "In the charter" (were you 
>>thinking to keep it or drop it?).  Then Extech would say, "A new or 
>>rechartering group must document the level-3 commitment in its 
>>charter.  An existing group may satisfy this checkpoint in one of the 
>>following ways [...this is yet tbd by us...].
>>
>>>And then we can
>>>point to it from the Notes to the corresponding Checkpoints in the Gd1
>>>and 2...
>>
>>We're going to have bi-directional links on *every* checkpoint, between 
>>the OpsGuide document and the OpsExtech document, just like the WAI 
>>standards.  (I'm planning some backward compatible extensions to our 
>>framework document grammar and our transformation stylesheets that would 
>>automate this.)
>>
>>-Lofton.
>>
>>
>>>-----Original Message-----
>>>From: Lofton Henderson [mailto:lofton@rockynet.com]
>>>Sent: Thursday, April 18, 2002 4:27 PM
>>>To: Kirill Gavrylyuk; www-qa-wg@w3.org
>>>Subject: RE: Ops Guidelines Issue
>>>
>>>
>>>We haven't addressed and answered the underlying issue yet -- do we
>>>intend
>>>to prescribe for existing WGs?  But for the sake of discussion, let's
>>>assume that the answer is "yes".
>>>
>>>Gd.1 and Gd.2 contain 7 checkpoints.  The first 6 apply to all WGs, new
>>>groups and existing groups.  *What* they should do is the same (if you
>>>ignore the words "in the charter").  The only difference is *how*, i.e.,
>>>
>>>the technique.  New group:  "In the charter".  Existing group:
>>>"Somehow"
>>>(okay, "somehow" could be something like "Minuted resolution in a WG
>>>meeting", or "charter amendment", or ...).
>>>
>>>Gd.7, by comparison, is only applicable to some WGs in some special
>>>circumstances -- externally developed TM which are to be transferred to
>>>the WG.
>>>
>>>I think that it is more natural and more streamlined to differentiate
>>>new/existing in the "Techniques", than to replicate a handful of
>>>checkpoints that specify essentially the same "what-to-do", with
>>>slightly
>>>different wording for new/existing.
>>>
>>>-Lofton.
>>>
>>>At 03:35 PM 4/18/02 -0700, Kirill Gavrylyuk wrote:
>>> >Why not to deal with this issue the same way we did for the Transfer of
>>>
>>> >the test suite from external party? (G7). In the Gd7 we just repeat the
>>>
>>> >applicable chkpt from the Gd1 and Gd2. We can insert
>>> >Gd 3 Introduce QA into existing WG
>>> >and put there applicable reworded checkpoints from Gd1 and Gd2 -
>>>similar
>>> >to Gd 7.
>>> >
>>> >What do you think?
>>> >
>>> >-----Original Message-----
>>> >From: Lofton Henderson [mailto:lofton@rockynet.com]
>>> >Sent: Thursday, April 18, 2002 3:26 PM
>>> >To: www-qa-wg@w3.org
>>> >Subject: Re: Ops Guidelines Issue
>>> >
>>> >
>>> >I have some thoughts now about how we could approach Issue #60.  I'll
>>> >omit detail for now, and focus on the overview:
>>> >
>>> >Overview&Proposal
>>> >--------
>>> >
>>> >Guideline 1 and Guideline 2 are all stated in terms of "In
>>> >Charter,...", so they are apparently only applicable to new groups.
>>> >But we say in 1.3, and
>>> >I believe that it is our intent, that *all* groups have QA
>>> >responsibility
>>> >-- new WG, WG in progress on first Rec, WG finished first Rec and
>>> >working
>>> >on subsequent one.
>>> >
>>> >If this latter principle were agreed, then we should reword Gd.1, Gd.2,
>>>
>>> >and their checkpoints.  They would not say "In charter", but rather
>>> >their wording would be applicable to all groups.  Then in two places we
>>>
>>> >could say
>>> >how it affects groups in various stages, and how the various WGs
>>>satisfy
>>> >
>>> >the checkpoint:
>>> >
>>> >1.) in the descriptive prose following each guideline and checkpoint;
>>> >
>>> >2.) and, in Ops-Extech we would distinguish and describe how WGs at
>>> >different stages satisfy the checkpoint:  new groups, "In charter";
>>> >other groups ... (some other way, e.g., minuted resolution in
>>> >face-to-face or teleconference, etc).
>>> >
>>> >Underlying Issue
>>> >-------
>>> >
>>> >The real question that needs to be answered before we implement such a
>>> >proposal is:  do we (QAWG) intend to assert that existing WGs have some
>>>
>>> >QA responsibilities, i.e., are ultimately responsible for production
>>> >and existence of test materials related to their standards?  Or do we,
>>> >as now,
>>> >intend to write prescriptions (re. commitment and resource allocation)
>>> >only
>>> >for new WGs and advise the rest to "review and consider incorporating
>>> >...etc..." (current sec 1.3).  This might be a delicate question -- an
>>> >existing WG may feel that, when its charter was approved, it had a
>>> >contract
>>> >for the scope of its work and required deliverables.
>>> >
>>> >Any thoughts on this?
>>> >
>>> >-Lofton.
>>> >
>>> >[issue#60] http://www.w3.org/QA/WG/qawg-issues-html.html#x60
>>> >
>>> >At 11:59 AM 4/15/02 -0600, Lofton Henderson wrote:
>>> > >QA Working Group --
>>> > >
>>> > >I have come up with an issue about the Ops Guidelines [1].
>>> > >
>>> > >Recently I have been looking at the QA aspects of SVG (while
>>> > >generating content for Ops-Extech), and have been looking at some
>>> > >existing
>>> >activities
>>> > >that have published Recommendations (such as XML 1.0 and XSLT 1.0).
>>> > >
>>> > >Issue:  Checkpoints don't clearly address existing groups.
>>> > >
>>> > >Description:
>>> > >
>>> > >In the introductory section 1.3, "Navigating..", we say:
>>> > >
>>> > >"This document is applicable to all Working Groups, including those
>>> > >that are being rechartered or already exist. Working Groups may
>>> > >already be doing some of these activities and should review the
>>> > >document and in so
>>> >
>>> > >far as possible incorporate principles and guidelines into their
>>> > >work"
>>> > >
>>> > >The first couple of guidelines -- QA responsibility, QA commitment,
>>> > >resource allocation, etc -- are all written for new groups.  There is
>>> >no
>>> > >mention of how an existing group should make its commitment, the TS
>>> > >responsibilities of a group that has published a Rec and has
>>> >rechartered
>>> > >or is rechartering.  For example:
>>> > >
>>> > >** in-progress towards Recommendation, but already chartered (e.g.,
>>> > >XFORMS)?
>>> > >
>>> > >** done w/ a first Recommendation, but moving on to further work
>>> > >(e.g., SVG, XSLT, XML)?
>>> > >
>>> > >Imagine being a member of one of these groups and looking at the
>>> > >first couple of Guidelines/Checkpoints.  What would you conclude
>>> > >about what
>>> >you
>>> > >should do?  I don't have a proposal yet, but one or more of the
>>> >following
>>> > >options might be appropriate:
>>> > >
>>> > >a.) reword the guidelines and checkpoints, or add new ones (i.e.,
>>> > >there would be "applicability" here -- some ckpts apply to new groups
>>>
>>> > >and
>>> >some
>>> > >to old groups).
>>> > >b.) add prose addressing "old groups"
>>> > >c.) add new/old criteria to Ops-Extech for pass/fail ("verdict
>>> >criteria")
>>> > >
>>> > >I think this is important enough that we should take a little time,
>>> > >so I'll log it as an issue, unless anyone objects.  (Btw, I'll have
>>> > >new, substantially revised issues list out today.)
>>> > >
>>> > >-Lofton.
>>> > >
>>> > >[1] http://www.w3.org/QA/WG/2002/framework-20020405/qaframe-ops
>>> > >
>
Received on Monday, 22 April 2002 14:16:44 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:09 GMT