W3C home > Mailing lists > Public > public-webrtc@w3.org > December 2017

Re: WPT test dependencies

From: Bernard Aboba <Bernard.Aboba@microsoft.com>
Date: Fri, 1 Dec 2017 04:14:22 +0000
To: Nils Ohlmeier <nohlmeier@mozilla.com>
CC: "public-webrtc@w3.org" <public-webrtc@w3.org>
Message-ID: <48F369A2-83AD-4232-9E1F-A0B303D556DE@microsoft.com>
On Nov 30, 2017, at 5:41 PM, Nils Ohlmeier <nohlmeier@mozilla.com<mailto:nohlmeier@mozilla.com>> wrote:

I guess this comes down to the ultimate question: is every implementation expected to implement all aspects of the spec?

[BA] It seems to me that a test should test the API it is focussed on, with as few deoendencies as possible. That isn’t a judgement about conformance criteria so much as a desire to easily figure out exactly what is broken when something regresses.

For example, if IceTransport and DtlsTransport depend on SctpTransport, then if SctpTransport regresses (or isn’t yet implemented) the other tests go red even if the IceTransport and DtlsTransport functionality is fine. That seems unhelpful to me, particularly if the end goal is to achieve and maintain a high level of interoperability with a large test matrix, where spurious results could deluge developers with misinformation.
Received on Friday, 1 December 2017 04:14:53 UTC

This archive was generated by hypermail 2.3.1 : Friday, 1 December 2017 04:14:54 UTC