Question on Developing Test Assertions

Now that we've finished going through the test assertions I generated for 
SpecGL, it seems there is still one unanswered question: How are we going 
to develop test assertions (and modify the existing SpecGL assertions) in 
the future?

There are at least 2 possible scenarios:

1) Develop the test assertions the same way I developed the SpecGL ones 
(i.e., use the conformance requirements PLUS additional info from 
Rationale, Discussion, Examples, etc.)  I used this additional info because 
I felt more info was needed to fully define (and test) the conformance 
requirements.

2) Use ONLY the conformance requirements - no more no less - to generate 
the assertions.  In my mind, this is the ideal, and preferred, 
approach.  The conformance requirements section SHOULD (MUST?) be complete 
and self-contained.  Using any additional info really means we're adding 
non-normative requirements to the test assertions.  However, in order to 
use this approach, the conformance requirements have to be scrutinized to 
ensure completeness.

Perhaps, there is a 2-tiered approach:  Use the additional info (scenario 
1) the first time the assertions are generated.  The main intent would be 
to flush out problems/inconsistencies in the guideline.  Then use ONLY the 
conformance requirements (scenario 2) to re-do the assertions for the final 
version.

Any thoughts?

Mark

****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
****************************************************************

Received on Tuesday, 3 February 2004 12:43:25 UTC