W3C home > Mailing lists > Public > www-qa-wg@w3.org > April 2002

Re: New Definitons for Glossary

From: Mark Skall <mark.skall@nist.gov>
Date: Mon, 15 Apr 2002 15:27:09 -0400
Message-Id: <5.0.0.25.2.20020415150605.01b98cb0@mailserver.nist.gov>
To: Lofton Henderson <lofton@rockynet.com>
Cc: www-qa-wg@w3.org
At 12:46 PM 4/15/02 -0600, Lofton Henderson wrote:
>Thanks for this, Mark...
>
>At 11:28 AM 4/12/02 -0400, Mark Skall wrote:
>>[...] I've provided some definitions for these terms.  I couldn't find 
>>any formal definitions cited elsewhere, but these are how we generally 
>>use these terms at NIST.  If anyone would like to modify these 
>>definitions, please respond to the e-mail.  If there are no suggested 
>>modifications, Karl, please enter these into the glossary.
>>
>>Generally, test assertion, semantic requirement and test requirement are 
>>used interchangeably.
>
>This is one point that interested me, because sometimes I suspect that 
>they are being used for different things (e.g., one refers to an 
>identifiable actual statement or statements in the specification, and 
>another refers to a "testable assertion" that is synthesized from one or 
>more statements in the spec).


No, the way we use them, they all refer to testable 
assertions.  "Requirement" by itself typically refers to a statement in the 
specification but we use "test requirement" or semantic requirement" to 
mean a testable assertion.



>>Test Assertion
>>A set of premises that are known to be true by definition in the spec.
>
>I think it would be helpful if the definition were less terse.  What does 
>one actually look like (e.g., example)?


A test assertion could be almost anything. The style and the syntax is 
quite different depending upon who develops them.  I've provided 2 examples 
from some test assertions we're developing to test a Smart Card 
specification.  The first is for a "normal" case and the second is for an 
"error" case that we've discussed in the last e-mail thread.

Assertion 2.1

Purpose: To test gscBsiUtilCardConnect() using a good card inserted into a 
specified reader.

Scenario:
1.      A valid card is in a particular reader available to the candidate SPS.
2.      A gscBsiUtilCardConnect() call is made to the SPS, with
•       uszReaderName == a pointer to a string containing the reader’s name
•       unReaderNameLen == an unsigned long variable containing the length 
of the reader’s name
•       hcard == a pointer to the unsigned long variable hcard1.

Expected Results:
3.      The call returns
•       the return code BSI_OK
hcard1 == a valid handle.


Assertion 2.6

Purpose: To test gscBsiUtilCardConnect() using a bad inserted card.

Scenario:
1.      A bad card is in a particular reader available to the candidate SPS.
2.      A gscBsiUtilCardConnect() call is made to the SPS, with
•       uszReaderName == a pointer to a string containing the reader’s name
•       unReaderNameLen == an unsigned long variable containing the length 
of the reader’s name
•       hcard == a pointer to the unsigned long variable hcard1.

Expected Results:
3.      The call returns
•       the return code BSI_CARD_ABSENT (??).





>How does a TA relate to actual identifiable content in a specification, etc.


It is derived from the requirement in the spec and should be traceable back 
to that section of the spec.

>-Lofton.

****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
**************************************************************** 
Received on Monday, 15 April 2002 15:23:01 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:09 GMT