RE: QA Glossary

On Fri, 24 May 2002, Arnold, Curt wrote:

> I'll go ahead.  I'd be extremely careful about how this is worded
> so that you don't discourage tests that attempt to reach one
> particular state and then make multiple assertions on query
> operations on that state.

I agree that the terminology should not discourage or encourage.
 
> A test should try to get a system to one particular state, but it
> is only beneficial from a code maintenance and coverage
> perspective to make as many observations and assertions at that
> final state as possible.

Depending on your definition of "as possible", the above statement may
not be true in general: Depending on the device under test, a "state"
may be volatile, so increasing the number of test cases that reuse
past state may not be a good idea, especially if you want a robust
test suite with repeatable results. 

For example, consider a Web caching proxy and test cases that probe
for stored content. The more you probe and the longer you test, the
higher is the chance that the proxy will silently purge a previously
cached document.

Moreover, I disagree that code maintenance and coverage perspective
have much to do with reusing past state. Depending on the design, code
maintenance can be actually simplified by _not_ reusing states. And
coverage itself is independent from whether you reuse old states:
coverage only depends on what rules are tested, not how much time or
how many test cases it takes to test those rules.

In summary, I think we should not be encouraging or discouraging a
particular test suite design because different users have different
suite requirements. We should limit ourselves to terms and statements
that remain valid regardless of the particular protocol, standard, or
suite user.

Alex.

Received on Friday, 24 May 2002 16:21:29 UTC