Re: testability definition

-- for those who are tired of long threads, there is a summary
   at the end of the message :-).

On Sun, 8 Sep 2002 skall@nist.gov wrote:

> No, this is not a trick.  Just like in testing, one can only test
> exactly what the spec says.  Likewise, we are only allowed to
> analyze what our spec says - ceratinly not what you think it says.
> Again, what it says is "check".
>
> Of course, thee are many specs that are not testable with the
> "check" phrase.  The whole point is that requirements can't be
> "checked" if they are not precisely defined [...] If a requirement
> is ambiguous, contradictory, or just plain vague it can't be
> checked.

You MUST define "to check that a requirement has been met" then! Your
explanations so far lead me to believe that you are ignoring the "has
been met" part of the phrase. I agree that requirements should be
defined, but it's not enough to satisfy the curent testability
definition, in my interpretation of it.

I am not sure why you are so confident that your interpretation of the
wording is the only possible one.  Unless you consider my concerns
totally bogus, clearly there is a lack of clear/unambigius
interpretation of the current wording. If my concerns are indeed 100%
bogus, I am somewhat surprised they got other people on the list to
ponder about the current definition.

> A requirement can't be "checked" unless it is precisely defined.

Depends on one's definition of "check", as you have illustrated
yourself. Nothing in the testability definition implies high precision
of requirement's definition. What is far more important is the "has
been met" part though.

To me, "to check that A HAS BEEN MET" in the context of the
testability definition means to produce yes/no answer (as a result of
a check). To you, it means to perform an undefined act of checking
(possibly without a result) when A is precisely defined. To somebody
else, it may mean to perform checking (possibly without a result)
regardless of the precision of A's definition. And so on... All those
interpretations seem to be possible. Thus, the testability definition
itself is imprecise and needs improvement.

If your intent is to require that all spec requirements are precisely
defined, I do not understand why introduce the definition of
testability. Again, it adds no value in that case! Why not simply say
that "spec MUST define all requirements" or something like that
instead of introducing testability definition?


To summarize,
	- you are assuming your interpreation is the only
	  possible one; this discussion implies it may not be
	- your explanations seem to ingore the "has been met" part
	- this thread proves, IMO, that the definition is
	  imprecise/ambiguous
	- until the ambiguity is resolved, it is impossible to
	  say whether behavioral specs are covered by the
	  testability definition

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance

Received on Monday, 9 September 2002 00:01:43 UTC