- From: Philipp Hancke <fippo@goodadvice.pages.de>
- Date: Wed, 31 Jan 2018 07:34:40 +0100
- To: "public-webrtc@w3.org" <public-webrtc@w3.org>
Am 30.11.2017 um 18:43 schrieb Bernard Aboba: > The web-platform-tests dashboard (see: https://wpt.fyi/webrtc ) contains a disclaimer that it does "not contain useful metrics for evaluation or comparison of web platform features". > > That said, the RTCIceTransport.html test does not succeed even on browsers that support the RTCIceTransport (e.g. Safari). > > The test is failing because this test (and the RTCDtlsTransport test ) depend on the SctpTransport object: > > "ReferenceError: RTCSctpTransport is not defined" > > Why does the RTCIceTransport test depend on the RTCSctpTransport being defined? > > For example, if a browser supports RtpSender/Receiver, DtlsTransport and IceTransport but not SctpTransport couldn't an RTCIceTransport object be obtained from sender.transport.transport ? So the second page here have a couple of problems: https://wpt.fyi/webrtc/RTCConfiguration-bundlePolicy.html claims Chrome does not support a balanced bundlePolicy or max-bundle which is not true from my experience. The test says: new RTCPeerConnection({ bundlePolicy: 'balanced' }) should succeed but attempts to use getConfiguration. The idiomatically correct test for this is to test that new RTCPeerConnection({bundlePolicy: 'balanced'}) does not throw. The current test fails for unrelated reasons. It also fails in Edge because Edge's native does not like being called without iceServers. None of this is mentioned in the review of the pull request. https://github.com/w3c/web-platform-tests/pull/6478 which makes me concerned about the review process. A good test for bundlePolicy is *much* harder to write. It has to make assertions about the type of ice candidates emitted. This problem of test style persists on a lot of pages: https://wpt.fyi/webrtc/RTCConfiguration-iceCandidatePoolSize.html https://wpt.fyi/webrtc/RTCConfiguration-iceServers.html https://wpt.fyi/webrtc/RTCConfiguration-iceTransportPolicy.html https://wpt.fyi/webrtc/RTCConfiguration-rtcpMuxPolicy.html In the last set of tests Firefox fails because of the lack of setConfiguration in the constructor. I did not bother to look much further but randomly poking around found this gem: https://wpt.fyi/webrtc/RTCPeerConnection-createOffer-offerToReceive.html fails in Edge because its native RTCPeerConnection does a silly thing Chrome used to do and throw on new RTCPeerConnection(). Again, trivial fix, just call new RTCPeerConnection(null) Writing good tests is hard. There are some quirks required to make them work in all browsers, the current approach of failing without need is not useful at all.
Received on Wednesday, 31 January 2018 06:35:35 UTC