W3C home > Mailing lists > Public > www-qa@w3.org > May 2003

Re: OpsGL QA-commitment-group

From: Mark Skall <mark.skall@nist.gov>
Date: Fri, 09 May 2003 11:03:54 -0400
Message-Id: <>
To: Lofton Henderson <lofton@rockynet.com>, Alex Rousskov <rousskov@measurement-factory.com>
Cc: www-qa@w3.org

At 08:46 AM 5/9/2003 -0600, Lofton Henderson wrote:
>At 09:56 AM 5/9/2003 -0400, Mark Skall wrote:
>>At 12:17 AM 5/8/2003 -0600, Lofton Henderson wrote:
>>>At 08:54 AM 5/7/2003 -0600, Alex Rousskov wrote:
>>>>On Wed, 7 May 2003, Lofton Henderson wrote:
>>>> > CP1.3: Commit to complete test materials. [Priority 3] Conformance
>>>> > Requirements: the WG MUST commit to produce or adopt a complete Test
>>>> > Materials before Recommendation, where complete is defined as: at
>>>> > least one test case for every identifiable conformance requirement
>>>> > of the specification.
>>>>I apologize if I missed the discussion about it, but not all
>>>>"identifiable conformance requirements" are testable and, hence, can
>>>>have at least one test case. Should "testable" qualifier be added?
>>>I don't have a problem adding such a qualifier.  Objections anyone?
>>I object.  The reason is that I don't accept Alex's premise.  Every 
>>requirement should (MUST) be testable.
>To clarify, are you saying:
>1.) that "testable" is part of the our definition of conformance 
>requirement (or should be), and therefore this is inherently redundant?

It certainly should be.  It's probably only implicit now, but should be 
made explicit.

>2.) or, that we should have (e.g., in SpecGL) a requirement (i.e., 
>checkpoint) that every conformance requirement in a target spec must be 

I would definitely welcome such a checkpoint.

>Note.  "Conformance requirement" is not in QA-glossary, nor in SpecGL "4. 
>Note2.  I recall a while back that we discussed the difference between 
>conformance requirement and test assertion in telecon, and someone 
>(Andrew?) drew a distinction that we liked, and Mark did an AI to take a 
>sample GL and show what its *test assertions* would look like, as opposed 
>to its conformance requirements.

Yes.  As I recall, the distinction was that requirements had MUSTs and test 
assertions were statements of present tense.

>>(In fact, I thought this statement was included somewhere in our guidelines)
>Not yet.  Or if we have addressed it, I can't remember where it is, off 
>the top of my head.  We have talked a lot about "test assertions", but 
>little about "requirements".
>There is a SpecGL issue group about TA-req.  As I recall, at least one LC 
>commenter noted in passing the lack of definition of "conformance 
>requirement".  (Or maybe I did -- but if you look at the issue group, then 
>you'll see that we need to agree about it before resolving the questions 
>about test assertions.)
>>  If  a requirement is not testable, it should be reworded to be testable 
>> or be eliminated from the specification.  If it can't be tested, it 
>> can't be verified that it was done correctly and is, thus, of no 
>> use.  Adding the suggested qualifier would sanction having non-testable 
>> requirements.
>Since there is dissent on this, I would propose that we close it in the 
>OpsGL context for now, *without* the suggested change.
>We will sort it out with the SpecGL TA-req issues group, and if that 
>implies any adjustment to OpsGL is appropriate, then we will do it.  (I'll 
>link this thread from that issue group, to make sure we don't overlook 
>it.  So ... continue ...)

Okay with me.  But I would like to see the testable requirement part made 
explicit, either through a new checkpoint or by adding clarifying words.


Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
Received on Friday, 9 May 2003 11:10:26 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:32 UTC