W3C home > Mailing lists > Public > www-qa@w3.org > November 2003

Re: lack of testability definition

From: Karl Dubost <karl@w3.org>
Date: Mon, 3 Nov 2003 15:38:12 -0500
To: www-qa@w3.org
Message-Id: <A4C8472A-0E3D-11D8-8488-000A95718F82@w3.org>


thank you for this definition.

I think that we are reaching the disagreement I was feeling at the last 
F2F meeting about testability. It might be that my understanding of 
testability is based on another assumption.

For me, I would say that everything is testable without ambiguity. I 
would say that every tests have a 3 possible states.

	Yes, No and Not Applicable.

You have to be sure when you design the test that "not applicable" is 
not part of the nature of the test itself, but one of the possible 

Le Lundi, 27 octo 2003, à 18:23 America/Montreal, Alex Rousskov a écrit 
> 	Informally, "boolean condition X is testable" usually means
> 	"there exist a procedure to determine the truthfulness of X".
> 	A more specific (and less general) wording may require the
> 	"procedure" to be "finite" or even "affordable". The problem
> 	with this informal approach is that for most practical
> 	purposes related to computers, it is impossible to determine
> 	truthfulness by following a procedure. The best we can do is to
> 	attain a high level of confidence that X is true. Thus,

For me, the assertion, you made in this paragraph is false. In the 
sense that the computers are not less testable than physics 
phenomenons. (Here, I have chosen Physics on purpose, that's my 
background and you have uncertainty laws like the position of an 
electron around an hydrogen atom).

In the assertion made here, you just show that you test the wrong thing 
and it's usually an error that many people do. :)

In Physics, you can't know the position of an electron around a proton 
in an hydrogen atom. But quantum physics gives you a framework which 
gives you the probability of finding the electron at a certain position.

You can't test the exact position, so it's not a test.
But you can test if the series of measurements respect the physics law.

The nature of testability is not intrinsec to the topic itself 
(computers, physics, etc) but to the way you define your tests.

So for me, there's no such thing like lack of testability. Everything 
is testable, it only depends on the way you define your tests. If the 
results of the test is unpredictable, it's because your test assertion 
has an issue.

Let's come back to a W3C specification.


	the value of the alt attribute specifies a text of replacement when 
the image is not displayed.

	If your test criteria, is that, the content of the alt attribute must 
be identically the same. The test is not good. It doesn't mean that the 
alt attribute is not testable, but that your test criteria is not well 

I'm not argueing about the definition, but more about the door we are 
opening. If we define things like not testable, it means that people 
will come with features in specification which are not testable and 
declared as it, which is dangerous.

Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager
*** Be Strict To Be Cool ***
Received on Monday, 3 November 2003 19:03:47 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:34 UTC