Re: OpsGL QA-commitment-group

At 08:46 AM 5/9/2003 -0600, Lofton Henderson wrote:
>At 09:56 AM 5/9/2003 -0400, Mark Skall wrote:
>
>>At 12:17 AM 5/8/2003 -0600, Lofton Henderson wrote:
>>
>>>At 08:54 AM 5/7/2003 -0600, Alex Rousskov wrote:
>>>
>>>
>>>>On Wed, 7 May 2003, Lofton Henderson wrote:
>>>>
>>>> > CP1.3: Commit to complete test materials. [Priority 3] Conformance
>>>> > Requirements: the WG MUST commit to produce or adopt a complete Test
>>>> > Materials before Recommendation, where complete is defined as: at
>>>> > least one test case for every identifiable conformance requirement
>>>> > of the specification.
>>>>
>>>>I apologize if I missed the discussion about it, but not all
>>>>"identifiable conformance requirements" are testable and, hence, can
>>>>have at least one test case. Should "testable" qualifier be added?
>>>
>>>I don't have a problem adding such a qualifier.  Objections anyone?
>>
>>I object.  The reason is that I don't accept Alex's premise.  Every 
>>requirement should (MUST) be testable.
>
>To clarify, are you saying:
>
>1.) that "testable" is part of the our definition of conformance 
>requirement (or should be), and therefore this is inherently redundant?


It certainly should be.  It's probably only implicit now, but should be 
made explicit.


>2.) or, that we should have (e.g., in SpecGL) a requirement (i.e., 
>checkpoint) that every conformance requirement in a target spec must be 
>testable?


I would definitely welcome such a checkpoint.


>Note.  "Conformance requirement" is not in QA-glossary, nor in SpecGL "4. 
>Definitions".
>
>Note2.  I recall a while back that we discussed the difference between 
>conformance requirement and test assertion in telecon, and someone 
>(Andrew?) drew a distinction that we liked, and Mark did an AI to take a 
>sample GL and show what its *test assertions* would look like, as opposed 
>to its conformance requirements.


Yes.  As I recall, the distinction was that requirements had MUSTs and test 
assertions were statements of present tense.

>>(In fact, I thought this statement was included somewhere in our guidelines)
>
>Not yet.  Or if we have addressed it, I can't remember where it is, off 
>the top of my head.  We have talked a lot about "test assertions", but 
>little about "requirements".
>
>There is a SpecGL issue group about TA-req.  As I recall, at least one LC 
>commenter noted in passing the lack of definition of "conformance 
>requirement".  (Or maybe I did -- but if you look at the issue group, then 
>you'll see that we need to agree about it before resolving the questions 
>about test assertions.)
>
>>  If  a requirement is not testable, it should be reworded to be testable 
>> or be eliminated from the specification.  If it can't be tested, it 
>> can't be verified that it was done correctly and is, thus, of no 
>> use.  Adding the suggested qualifier would sanction having non-testable 
>> requirements.
>
>Since there is dissent on this, I would propose that we close it in the 
>OpsGL context for now, *without* the suggested change.
>
>We will sort it out with the SpecGL TA-req issues group, and if that 
>implies any adjustment to OpsGL is appropriate, then we will do it.  (I'll 
>link this thread from that issue group, to make sure we don't overlook 
>it.  So ... continue ...)
>
>Okay?


Okay with me.  But I would like to see the testable requirement part made 
explicit, either through a new checkpoint or by adding clarifying words.


>-Lofton.
>

****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
****************************************************************

Received on Friday, 9 May 2003 11:10:26 UTC