Re: NIST comments on TestGL

Just a couple observations about how Dimensions of Variability (DoV)
affect TestGL.

>4. Ckpt 1.4 Group test assertions by levels, profiles, modules, etc
>a) Actually, test assertions should be grouped by what makes sense,
>e.g., grouped by interface. It would be helpful to explain the
>rationale/benefit for this ckpt...
>c) Many people may be unfamiliar with Degree of Variability  so, need
>to link to its discussion. Or, can we remove this notion? Does it add
>to the understanding and/or implementation or verification of this
>checkpoint?

While grouping the assertions might not be a goal in itself, it is
crucial to use DoV in organizing the tests so that when you actually
test a product, you use the subset of the test suite that applies to
the modules, levels, discretionary choices, etc. implemented by that
product, plus the spec version, of course.

>6. Ckpt 1.7: Identify optional behaviors in the specification

This should be tied in with the above. Has the QAWG reached a
conclusion about how to define "discretionary behaviors"? I think
that optional behaviors are modules or levels when they are bundles
of functionality, otherwise they are discretionary behaviors. Then
we have "discretionary choices" as a subset: the spec provides an
enumerated set of choices, and no other behavior conforms. For
example, support for a particular natural language is likely to be
a discretionary behavior, but not a discretionary choice item, but
typically not a module either.
.................David Marston