Reconciling multiple proposals [was Re: Observations about TestGL organization]

David, thanks for this carefully reasoned analysis of TestGL and proposal 
for re-organizing the GL and CP.

All... we have two proposals for re-organizing.  There are certain things 
in common (esp. thoughts about major groupings of old GL & CP) and some 
differences as well (esp. order, and slightly different groupings).  It 
seems to me that we ought to take one proposal as base, and use the other 
proposal to generate any fundamental issues against base.

Given that Mark and Kirill have both generated specific comments against 
the Peter/Patrick proposal, it might be sensible to use that as base.

What do you think (all)?

-Lofton.

p.s.  As I read David's proposal, I tried to correlate major groupings to 
Peter/Patrick proposal.  I.e., where in P/P proposal is David's concept 
mostly addressed.  This is quick (and may reflect some misunderstandings of 
the proposals).  Specific comments....

At 04:44 PM 3/8/03 -0500, David Marston/Cambridge/IBM wrote:

>This message summarizes my overall view of the Test Guideline (GL) and
>Checkpoint (CP) organization. I hope to send along some more detailed
>ideas later.
>
>The CPs have to be flexible because different WGs can approach the
>whole testing strategy in different ways, and they may choose different
>levels of commitment. The Commitment Table at [1] has more levels than
>checkpoints, which becomes useful in this discussion of TestGL. In
>other words, if a WG commits to a QA level that is between two OpsGL
>checkpoints, it still helps them meet more TestGL CPs. Notably, test
>cases may be written by the WG or contributed, or both, which ties to
>the difference between QA levels 3 and 4.
>
>Without intending to, I devised a new allocation of the CPs across 7
>new guidelines. Like the SpecGL guidelines, I have tried to arrange
>them in a logical progression.
>
>1. The current GL 7 could be split, with one part being a new GL 1 about
>planning. It would include old 7.1, 1.1, a revised 1.2 allowing test
>assertions (TAs) to be explicit or not, something about test case
>contributions, and maybe old 3.2.

P/P:  new GL7 & GL1

>Upon fulfilling the above guideline, the WG stands ready to divide the
>test materials work among planning the test-case management as static
>objects, planning the methods of running and reporting, generating a
>list of cases needed, and writing test cases.
>
>2. The next logical GL is probably old GL 1. CPs 1.3 through 1.8 cover
>the Dimensions of Variability (DoV) and belong here, as does old
>1.10. Old 1.9 belongs here too, but should be broadened to cover
>test planning and test case development as ways of checking the spec,
>especially in pre-CR stages.

P/P:  new GL2


>3. The next GL is old GL 6, since it's part of the planning and
>analysis phase. CPs 6.3, 2.2, and 4.14 all deal with resolving the
>match between tests and spec, whether through the intermediary of
>TAs or not, and should be refactored in any new arrangement of CPs.

P/P:  new GL4


>4. Then comes whatever is left of GL 3. Old CP 2.1 probably goes here
>as well. GL 2 is thus refactored away.

P/P:  new GL1


>Up to this point in the progression, the WG has identified "what
>tests ought to be written", though whether they attempted to make
>an exhaustive list depends on their committed QA level.
>
>5. Next comes that portion of GL 4 that deals with collecting test
>cases and managing their role and status. All the DoV should be
>addressed again, as shown by old CPs 4.6 to 4.10 (that parallel old
>1.3 to 1.8). An interesting new flag comes from old 1.4: tests can
>be flagged as conformance tests vs. interoperability tests, which
>allows useful contributions to be retained in a lesser status
>rather than rejected outright. Old CPs 4.12 and 4.4/4.13 (nearly
>the same?) also belong here.

P/P:  new GL3


>Up to this point in the progression, the WG has the plan and the
>methods for gathering and storing test cases. Notice that CPs
>(old) 1.4, 4.3, and 4.12 can be applied on an individual test
>case/assertion basis.
>
>6. The next GL should be that portion of old GL 4 that deals with
>running tests as opposed to writing them. Old CPs 4.2, 4.3, and
>4.5 go here. I'm not sure if CP 4.1 goes anywhere, but it could go
>here.


P/P:  new GL5


>7. The final GL takes old GL 5 and sharpens it a bit. All the CPs
>5.1 to 5.5 stay here, plus 4.11 and 7.2.

P/P:  new GL6


>This plan is very rough! I expect that CPs will change as they fit
>into this new progression.
>.................David Marston
>
>[1] http://www.w3.org/TR/qaframe-ops/#commitment-table

Received on Monday, 10 March 2003 10:54:26 UTC