Re: Question on Developing Test Assertions

We may have discussed this before, but I don't think we've reached 
agreement. (At least, I don't agree!)

The masses don't read specs. They are read be implementors of the 
technology and by people developing tests of those implementations. Both 
of these groups need precise formal language and if this means that the 
text is not "short and readable", so be it. In many (most?) cases the 
people developing tests, and therefore identifying assertions, typically 
aren't those who developed the spec. To ask them to "interpret the spec" 
and to write assertions in their own words rather than in the words of 
the spec authors opens up the possibility of diverging interpretations. 
I still think the ideal is for assertions to be identified directly 
within the spec by some kind of markup, which in some cases could be 
performed by the spec authors themselves.

As for "assertions are statement of fact about the implementation rather 
than requirements imposed", this distinction seems unnecessary to me. I 
don't really mind whether an assertion says "when you push the big red 
button the machine must halt" or "pushing the big red button halts the 
machine". Both carry the same meaning in the context of conformance. 
Requiring someone to go through the spec, copy the first form of the 
text, and then convert it into the second form seems like busy-work to me.


Mark Skall wrote:

>>
>> 2) In developing an assertion list, should the text from which they 
>> are drawn be "rewritten" to make it more formal and precise, or 
>> should we quote exactly the text from the spec?
>
>
>> I vote for the latter. This opens up the possibility of inserting 
>> markup directly into the spec to identify the assertions.
>>
>> Of course, if we simply do as we did for this exercise - insert 
>> additional (assertion) text into the spec that "interprets" other 
>> text, this defeats the purpose.
>
>
>
> I think we're straying from the issue here.  You've brought up a 
> different (but not really new) issue that we've discussed before.  My 
> position is that you need to keep the conformance requirements short 
> and readable by the masses.  The test assertions may be in more detail 
> and need to be read only by the test developer.  The test assertions 
> should make it precisely clear what the test will be and may be quite 
> long.  Additionally, Andrew (or perhaps someone else) introduced the 
> idea that the assertions are statement of fact about the 
> implementation rather than requirements imposed (e.g., the spec 
> contains rather than the spec MUST or the implementation produces a 
> file that . . . rather than the implementation MUST).  This may be a 
> subtle distinction but one that really makes a lot of sense (at least 
> to me).
>
>
> Mark
>
>
>
> ****************************************************************
> Mark Skall
> Chief, Software Diagnostics and Conformance Testing Division
> Information Technology Laboratory
> National Institute of Standards and Technology (NIST)
> 100 Bureau Drive, Stop 8970
> Gaithersburg, MD 20899-8970
>
> Voice: 301-975-3262
> Fax:   301-590-9174
> Email: skall@nist.gov
> ****************************************************************

Received on Wednesday, 4 February 2004 15:04:48 UTC