W3C home > Mailing lists > Public > public-test-infra@w3.org > July to September 2016

Tests that fail due to server problems

From: Mark Watson <watsonm@netflix.com>
Date: Tue, 13 Sep 2016 15:53:41 -0700
Message-ID: <CAEnTvdDq7hb-px58m8Ayoktke+m2BcOX5qawj-37xxMaHo9rGQ@mail.gmail.com>
To: public-test-infra <public-test-infra@w3.org>
We have some tests (for encrypted-media) which rely on a 3rd party server.
Presently, if that server fails then the tests report TIMEOUT, which is
then indistinguishable (for some of the tests) from certain kinds of test
failure (for example, video was expected to start but never did).

What is the appropriate result when the inability to complete the test is
due to such a 3rd party dependency, rather than a problem with the
implementation under test ? Should this be NOTRUN ? How do we trigger this ?

Received on Tuesday, 13 September 2016 22:54:12 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:12 UTC