Re: lack of testability definition

On Tue, 4 Nov 2003, Al Gilman wrote:

> At 11:29 AM 2003-11-04, Alex Rousskov wrote:
> >On Tue, 4 Nov 2003, Al Gilman wrote:
> >
> > > <draft
> > > class="strawman glossaryEntry notDefinition">
> > >
> > > A <term class="general">test</term> is a repeatable procedure for
> > confirming
> > > or denying the truth value of a proposition.
> > >
> > > A proposition is <term class="general">testable</term> if there is such a
> > > procedure that leaves little doubt as to the truth value of the proposition
> > > after the procedure has been applied.
> > >
> > > </draft>
> >
> >         A proposition is testable if there is such a procedure
> >         that assesses the truth value of a proposition with a high
> >         enough confidence level. Whether the confidence level is
> >         measurable and what confidence level is high enough
> >         depends on the proposition and its framework.
>
> That is interesting.  We are very close, but your restatement
> discloses a difference.  I don't feel that 'enough' belongs in a
> definition for our use of this term.  Because 'enough' is not, in
> general, defined.

Removing "enough" from the above text is acceptable to me. "High"
alone is sufficiently undefined to highlight the problem, IMO. As the
above text says, the definition of what is "high" depends on factors
beyond Glossary scope.

I agree with most of your observations regarding the [engineering]
goal of testing.

> The most glaring example of the non-boolean nature of test results
> comes from medical ethics, where the experimentation with a new
> clinical test may be terminated before the originally planned length
> of the trial because the evidence gathered to date demonstrates with
> sufficient confidence that deployment of the test will save lives.
> Even though it is at the same time demonstrated that sometimes it
> will return erroneous results.

I think you define test results broader than I would. You seem to
include the test-related decision making in the test result itself,
and that makes the test result non-boolean. Given your medical ethic
example above, the test assertion is something like:

	high [enough] number of people are likely [enough] to
	benefit from the drug while low [enough] number of people
	are likely to die sooner

If an early test yields a boolean "yes" answer with high enough
confidence, then the drug can be released and long-term tests can
continue concurrently. The decision making process here (defining
"highs" and "lows" and allowing the drug to be sold) is separate from
the testing itself, but may affect test duration and other testing
aspects, of course.

Testing is not equivalent to decision making. It is just a part of the
process.

Alex.

Received on Tuesday, 4 November 2003 14:06:18 UTC