- From: Alex Rousskov <rousskov@measurement-factory.com>
- Date: Wed, 3 Sep 2003 11:59:03 -0600 (MDT)
- To: Bjoern Hoehrmann <derhoermi@gmx.net>
- Cc: www-qa@w3.org
On Wed, 3 Sep 2003, Bjoern Hoehrmann wrote:
> The current Specification Guidelines document
> <http://www.w3.org/QA/WG/2003/08/qaframe-spec> does not define what
> it consideres "testable" or "measurable" while it spells out
> requirements that rely on these terms. Please explicitly define what
> is considered testable, especially in scope of conformance testing
> software. For example, if a specification reads, "The <address>
> element MUST contain contact information" I would not consider the
> statement testable.
IIRC, there was a long thread on this mailing list discussing this
very issue. There were at least two points of view:
1) It is impossible to define "testable" precisely.
Statement X is "testable" when there exists
a finite algorithm/procedure that correctly,
with sufficient probability, assesses
the truthfulness of X. What is sufficient
or even finite is impossible to define precisely.
2) It is possible to define what "testable" means.
No workable definition was provided though.
I favor (1).
I agree that many specs contain untestable statements. Moreover, I
assert that it is often impossible to write a spec without untestable
statements. Sometimes, untestable statements MUST be made and are
pragmatically useful (i.e., they help produce compliant
implementations).
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance
Received on Wednesday, 3 September 2003 13:59:17 UTC