Re: Spec Guideline: non-use of a DoV (re: issue #87, #94, etc.)

On Tue, 8 Oct 2002, David Marston/Cambridge/IBM wrote:

> And how does the reader ascertain that "modules are not specifically
> introduced/defined"?

From reading and analyzing the spec, I guess. I see no value in being
able to automatically determine whether "modules" or similar
abstractions are used.

> If the spec at least satisfies Checkpoint 10.2,
> then the reader can just read the conformance section rather than
> scan the entire document trying to find all the DoV that are in use.

Since the definition for most DoVs are so vague, enumerating them in a
conformance section adds little practical value, if any. One has to
understand the spec to actually make any important conclusions related
to DoVs. You want to keep DoV definitions vague to accommodate more
specs, but by doing so you also decrease the information behind a DoV
keyword like "module". As it has been said several times on this list,
a "module" can be "almost anything". If a conformance section says
that the spec contains or does not contain "almost anything", that is
not much information to go on. One has to read the spec.

> Still, I contend that an explicit declaration of non-use of each
> DoV is better than leaving the reader to determine the situation
> by scanning and finding nothing. A good compromise here is to
> require the conformance clause to state "all products must be the
> same except for the following permissible variations" and then list
> all allowable variability.

The latter is impractical for specs with many permissible variations
(or the authors would simply to cheat and group the variations
together, making the explicit listing not useful). Conformance
sections should be very short or they would need a meta-conformance
sections.

> There was a fine point in my earlier advocacy of particular DoV as
> opposed to allowing each WG to name and define their own forms of
> variability:
> >When you get down to details, the spec that employs a particular
> >DoV must be explicit in doing so. Having a limited set of 8 DoV
> >(with modules being so generic as to allow widespread variation)
> >helps those who want to write the ***testing framework.***
>
> In other words, allowing each WG to re-factor variability for
> their spec may not be too damaging to the test suite they produce
> (or cause to be produced) in association with their spec, but it
> places an enormous burden on those who will produce the testing
> framework.

Yes, a one-size-fits-all framework is difficult to produce when spec
authors are given a lot of freedom. The questions is, what comes
first: good specs or a one-size-fits-all framework? I argue that good
specs are more important and, hence, prefer to err on the side of
author's freedom. Again, I think we should dance from general "good
specs" requirements, not from "The One True Test Framework"
requirements.

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance

Received on Tuesday, 8 October 2002 01:23:14 UTC