W3C home > Mailing lists > Public > www-qa@w3.org > June 2003

Re: The bad side of test cases

From: Alex Rousskov <rousskov@measurement-factory.com>
Date: Mon, 23 Jun 2003 08:55:58 -0600 (MDT)
To: Karl Dubost <karl@w3.org>
cc: www-qa@w3.org
Message-ID: <Pine.BSF.4.53.0306230848540.29936@measurement-factory.com>

On Sun, 22 Jun 2003, Karl Dubost wrote:

> Dave Hyatt has added some comments today
> http://weblogs.mozillazine.org/hyatt/archives/2003_06.html#003542
> 	"""I think the solution is that the W3C needs
> 	to take an active role in producing test suites
> 	and - most importantly - provide guidelines
> 	 for proper reporting of results. It bothers me
> 	that drafts in the W3C get all the way to
> 	recommendation status without concrete test
> 	suites that can be used to gauge whether you
> 	have two or more interoperable implementations."""

The last phrase surprises me. Incompliant implementations may be
interoperable. Most test suites are designed to test whether an
implementation is compliant with some spec, not interoperable with
another implementation. While tests may contain robustness and
compatibility test cases (ours, for example, does), it should not be
W3C task to develop tests with such focus. W3C tests, if any, should
focus on compliance.


                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance
Received on Monday, 23 June 2003 10:56:01 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:32 UTC