Re: Thursday's Discussion

>This is a side issue, but related -- if we are going to look at evaluation 
>criteria such as "specific enough to generate tests", it helps to have a 
>concept of what tests for SpecGL might look like.
>
>What do you think of when you think of "tests"?
>
>As we discussed earlier (but I'm not sure of our level of consensus), a 
>manual test could be as simple as a per-conformance-requirement (or 
>per-TA) question, "Does the spec ...blah...?  Choice of:  NO; YES at 
>[...fill-in-link-to-spec-loc...]"  "Blah" is effectively the statement of 
>the conformance requirement.
>
>Are you thinking of manual testing like this?  Or are you thinking of some 
>sort of automated SpecGL tester?  Or ...?


I am thinking of a precise way to check to see if the requirement is 
met.  In the case of SpecGL, in my mind, this would be a completely manual 
process.  The key is precision, not automation.  If the requirements (and 
TAs) are not precise, you will end up with different interpretations and 
different tests.



>Can you also supply a list of the ones that you found to be problematic 
>(and why)?


Without going over the reasons in detail (we will do this at the telcon), 
the following are checkpoints where I needed more info or made assumptions 
in order to produce the assertions:

1.1, 1.2, 2.3, 4.1, 7.2, 8.2, 8.5, 9.2, 10.1, 10.2


Some of these are fairly trivial and some are a little more 
substantive.  However, as we all know, when it comes to conformance,  even 
the most trivial requirements must be precisely specified.




****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
****************************************************************

Received on Tuesday, 16 December 2003 16:01:51 UTC