RE: testability definition

FWIW, I'm fine with the current definition. I don't think there are any
serious flaws in it. 

Of course I have my preferences:
- it could be extended or clarified in the accompanying prose taking
some thoughts from these long threads. 
- I think ambiguity and under-specified behaviors should be addressed as
part of the testability requirement, at least in the prose.
- the "cost-effective finite" words could be taken away.
"cost-effective" is subjective, "finite" is IMHO unnecessary, since
we're trying to see if a 100% coverage is possible, but nobody intends
to cover 100% in practice.

but I'm not going to argue hard for them. I may suggest some additional
explanation text if I have time at some point, but may not get to it. 


Alex, I found your statement interesting:
> I do not need to prove that the process can be built. I need to apply
> an existing process to an existing implementation or document. I do
> not care if a spec is testable in some abstract way.
which I read that you don't care whether the spec is testable or not,
you care whether the particular implementation is testable. 

Those 2 are a bit different notions.  The spec is testable when:
- you can check all the requirements stated in it
- the requirements are unambiguous
- the behavior for every possible input is addressed. Explicit statement
"the behavior for input A is not defined" is one of possible ways
addressing input A.

One can build testable and not testable implementation for a testable
spec. A testable implementation does not mean the spec is testable
either. May be this could be even recorded in the Ex-Tech for the
SpecGL.


-----Original Message-----
From: Alex Rousskov [mailto:rousskov@measurement-factory.com] 
Sent: Friday, September 06, 2002 8:36 PM
To: Kirill Gavrylyuk
Cc: www-qa@w3.org
Subject: RE: testability definition

On Fri, 6 Sep 2002, Kirill Gavrylyuk wrote:

> True, this is an assumption I'm making. Do you have an example of
> the behavioral W3C spec that has non-enumerable input space (of
> power of continuum)?

SpecGL is, primarily, not about existing W3C specs but future specs so
existing examples have little value.

> You don't need to build and execute a process in order to prove that
the
> process can be built.

I do not need to prove that the process can be built. I need to apply
an existing process to an existing implementation or document. I do
not care if a spec is testable in some abstract way. I need to test
real implementations using real tools that detect real spec
violations.

What good does it make to know that a spec is testable if I cannot
test an implementation, and I cannot pay somebody to test it for me?!

> In fact some languages specs (like Java) were proven this way
> (although it took ~3 years for Java).

I do not need to prove that Java abstraction is a complete(?)
language identical to a Turing machine. I need to prove that a given
Java interpreter complies with Java specs. A much more practical task.

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance

Received on Monday, 9 September 2002 00:56:11 UTC