W3C home > Mailing lists > Public > www-qa@w3.org > September 2002

RE: testability definition

From: Kirill Gavrylyuk <kirillg@microsoft.com>
Date: Fri, 6 Sep 2002 16:31:38 -0700
Message-ID: <B3F0DACD72892E4DB7E8296C6C9FC2F6050753DA@red-msg-03.redmond.corp.microsoft.com>
To: "Alex Rousskov" <rousskov@measurement-factory.com>
Cc: <www-qa@w3.org>

> note that "infinity" does not assume "enumeration ability"
True, this is an assumption I'm making. Do you have an example of the
behavioral W3C spec that has non-enumerable input space (of power of
Schema's xsd:float doesn't count, since we're working with lexical and
not semantic space there.

> I would be against that because we should be talking about practical
things and infinite cost-ineffective processes are not.

You don't need to build and execute a process in order to prove that the
process can be built.
So theoretical testability is "practical" in many cases. Otherwise one
wouldn't be able to prove anything about sequences in math:)

In fact some languages specs (like Java) were proven this way (although
it took ~3 years for Java).

-----Original Message-----
From: Alex Rousskov [mailto:rousskov@measurement-factory.com] 
Sent: Friday, September 06, 2002 4:07 PM
To: Kirill Gavrylyuk
Cc: www-qa@w3.org
Subject: RE: testability definition

On Fri, 6 Sep 2002, Kirill Gavrylyuk wrote:

> Why not remove "finite cost-effective" and leave:
>        A specification is testable if there exists some
>        process with which a person or software can check
>        that the requirement has been met."

I would be against that because we should be talking about practical
things and infinite cost-ineffective processes are not. It simply does
not matter if such a process exists in theory because it would never
exist in practice.

Moreover, note that "infinity" does not assume "enumeration ability" -
- you cannot enumerate real numbers, for example. Thus, if a spec
includes an equivalence of a real number as an input, you cannot even
talk about making a process of enumerating valid inputs! I cannot
recall the math term for this phenomenon.

> IMO all we want to say is that a (behavioral) specification is
> testable if one can deduce a (unambiguous) behavior from it for any
> input allowed by the specification's grammar.

I almost agree with that. You need to be able to deduce behavior from
invalid input as well. Also, ambiguous behavior may be allowed by the
specs (e.g., it may be OK for HTTP proxy to cache the page or not to
cache it).

Your proposed definition is much more broad though.

> Then the proposed criteria is:
> The spec is testable as long as you can build the process (not
necessarily finite) which:
> -  enumerates all possible inputs (true in case of both HTTP and SOAP)

Not true if inputs are real numbers (at least).


                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance
Received on Friday, 6 September 2002 19:32:10 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:29 UTC