- From: Kirill Gavrylyuk <kirillg@microsoft.com>
- Date: Sun, 8 Sep 2002 21:55:38 -0700
- To: "Alex Rousskov" <rousskov@measurement-factory.com>, <skall@nist.gov>
- Cc: <www-qa@w3.org>
FWIW, I'm fine with the current definition. I don't think there are any serious flaws in it. Of course I have my preferences: - it could be extended or clarified in the accompanying prose taking some thoughts from these long threads. - I think ambiguity and under-specified behaviors should be addressed as part of the testability requirement, at least in the prose. - the "cost-effective finite" words could be taken away. "cost-effective" is subjective, "finite" is IMHO unnecessary, since we're trying to see if a 100% coverage is possible, but nobody intends to cover 100% in practice. but I'm not going to argue hard for them. I may suggest some additional explanation text if I have time at some point, but may not get to it. Alex, I found your statement interesting: > I do not need to prove that the process can be built. I need to apply > an existing process to an existing implementation or document. I do > not care if a spec is testable in some abstract way. which I read that you don't care whether the spec is testable or not, you care whether the particular implementation is testable. Those 2 are a bit different notions. The spec is testable when: - you can check all the requirements stated in it - the requirements are unambiguous - the behavior for every possible input is addressed. Explicit statement "the behavior for input A is not defined" is one of possible ways addressing input A. One can build testable and not testable implementation for a testable spec. A testable implementation does not mean the spec is testable either. May be this could be even recorded in the Ex-Tech for the SpecGL. -----Original Message----- From: Alex Rousskov [mailto:rousskov@measurement-factory.com] Sent: Friday, September 06, 2002 8:36 PM To: Kirill Gavrylyuk Cc: www-qa@w3.org Subject: RE: testability definition On Fri, 6 Sep 2002, Kirill Gavrylyuk wrote: > True, this is an assumption I'm making. Do you have an example of > the behavioral W3C spec that has non-enumerable input space (of > power of continuum)? SpecGL is, primarily, not about existing W3C specs but future specs so existing examples have little value. > You don't need to build and execute a process in order to prove that the > process can be built. I do not need to prove that the process can be built. I need to apply an existing process to an existing implementation or document. I do not care if a spec is testable in some abstract way. I need to test real implementations using real tools that detect real spec violations. What good does it make to know that a spec is testable if I cannot test an implementation, and I cannot pay somebody to test it for me?! > In fact some languages specs (like Java) were proven this way > (although it took ~3 years for Java). I do not need to prove that Java abstraction is a complete(?) language identical to a Turing machine. I need to prove that a given Java interpreter complies with Java specs. A much more practical task. Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance
Received on Monday, 9 September 2002 00:56:11 UTC