Re: QA Glossary - New version

On Thu, 23 May 2002, Mark Skall wrote:

> You're confusing the definition of "conformance" with the concept of 
> "proving conformance".

I do not think so. I do think that the current definition of
Conformance can be improved (see my original posting), but my
"uncertainty" questions are a completely different issue, not
dependent on the current wording of the Conformance definition. That
is, I argue that more definitions are needed for the terminology as a
whole to become practically useful for the purpose of conformance
testing and especially certification. Perhaps I demand too much from a
glossary document.

> Falsification testing can never prove conformance, only
> non-conformance (if an error is found).  In fact, when we advise
> bodies giving out certificates of conformance, we tell them to
> state that passing all the tests does not guarantee conformance.
> In fact, we say something like this in our QA guidelines.  Of
> course, the more test cases you have the better the chances are
> that passing all the tests results in conformance.

Sure. The problem is that if "certificate of conformance" does not
really imply conformance, we (a) should not promote that misleading
terminology and (b) should explicitly state somewhere that conformance
is not verifiable in general. That's ugly, but seems to be logical.

Alternatively, we could change the definitions so that certification
becomes more meaningful. For example, the current definition of
Certification ("A document or a service attesting the level of
conformity of the tested application") could be changed to something
like "A document or a service attesting absence of known conformance
violations within a given set of test cases".

My interest in this discussion is mostly pragmatic -- I want to
understand whether my test suite should generate "conformance
certificates" or some other kind of "quality mark" to comply with
future W3C QA standards/terminology.

Alex.

Received on Friday, 24 May 2002 00:07:06 UTC