Re: NIST comments on TestGL

At 12:18 PM 10/3/02 -0400, David Marston/Cambridge/IBM wrote:

>Just a couple observations about how Dimensions of Variability (DoV)
>affect TestGL.
>
> >4. Ckpt 1.4 Group test assertions by levels, profiles, modules, etc
> >a) Actually, test assertions should be grouped by what makes sense,
> >e.g., grouped by interface. It would be helpful to explain the
> >rationale/benefit for this ckpt...
> >c) Many people may be unfamiliar with Degree of Variability  so, need
> >to link to its discussion. Or, can we remove this notion? Does it add
> >to the understanding and/or implementation or verification of this
> >checkpoint?
>
>While grouping the assertions might not be a goal in itself, it is
>crucial to use DoV in organizing the tests so that when you actually
>test a product, you use the subset of the test suite that applies to
>the modules, levels, discretionary choices, etc. implemented by that
>product, plus the spec version, of course.

I agree, and it seemed that we were moving towards this conclusion in the 
recent TestGL-review telecon (9/25).


> >6. Ckpt 1.7: Identify optional behaviors in the specification
>
>This should be tied in with the above. Has the QAWG reached a
>conclusion about how to define "discretionary behaviors"? I think
>that optional behaviors are modules or levels when they are bundles
>of functionality, otherwise they are discretionary behaviors.

We have some work to do here with sorting out consistent terminology, 
especially between TestGL and SpecGL.

I would prefer that terms like "optional behaviors" not be used as you 
suggested.  Reason.  "Modules" is one of the DoV.  By their nature, the DoV 
are optional (being some of the principal ways in which specs may vary, at 
the option of authors and WGs).  If DoV remains in SpecGL as an organizing 
principal, then that needs to be reflected in TestGL (as you said initially 
above).

But in SpecGL (Guideline 8, "discretionary items", [1]), we have started to 
use "optional [features]" as a sub-class of the dimension.

>Then
>we have "discretionary choices" as a subset: the spec provides an
>enumerated set of choices, and no other behavior conforms.

GL8 [1] is only a first try at it, but the current draft does try to 
provide some clarification and classification about "discretionary xxx":

GL8:  Define discretionary items.
[they] come in three basic variants:
         1.) discretionary choices
         2.) optional features
         3.) implementation dependent values/features

>For
>example, support for a particular natural language is likely to be
>a discretionary behavior, but not a discretionary choice item, but
>typically not a module either.

Natural language ... isn't this usually dealt with in standards by 
referring to RFCxxxx, which defines the abbreviations for a whole slew of 
registered natural languages (plus a syntax)?  So in the case that it is 
required to use one of those, it would be a discretionary choice.  Else 
open-ended (e.g., if you could use your own made-up language tag), in which 
case you could argue that it  fits under "implementation dependent value".

-Lofton.


[1] http://www.w3.org/TR/qaframe-spec/#b2b3b3d481

Received on Thursday, 3 October 2002 13:41:54 UTC