W3C home > Mailing lists > Public > public-webrtc@w3.org > February 2018

Re: WPT test dependencies

From: Philipp Hancke <fippo@goodadvice.pages.de>
Date: Mon, 5 Feb 2018 19:03:34 +0100
To: public-webrtc@w3.org
Message-ID: <2d5696ca-6a6e-78be-fae6-7cea0455a091@goodadvice.pages.de>
> Even though we do not have the equivalent of block chain's smart
> contract, we can modify the commit / review / merge process to enforce
> it.

The current process had this gem slip through as well:
     https://github.com/w3c/web-platform-tests/pull/5847

In this particular case the spec was vague:
 
https://github.com/w3c/web-platform-tests/pull/5847/files#diff-908cb5d58eaaf36277efb47afe2fe6c6R120
and the "behave as if" resulted in the longest spec issue ever.

At the same time the test assertion
 
https://github.com/w3c/web-platform-tests/pull/5847/files#diff-908cb5d58eaaf36277efb47afe2fe6c6R136
was not passing in Chrome or Firefox or Edge. This should have raised a 
flag. It did not.

That test descriptions like "new RTCIceCandidate()" pass review is absurd.


A working review process would require to show which browsers a test 
passed in and discuss failures. If a test did not pass the ball goes to 
a person working on that browser for triage. This might result in a bug 
being filed against that browser or the spec.

Such a process is of course much more time-consuming and expensive -- 
unless you are trying to make this a "community effort".


 > Don't be shy, "Be a programmer" (TM).

looks like my last commit in the webrtc directory is more recent than yours.
Received on Monday, 5 February 2018 18:04:34 UTC

This archive was generated by hypermail 2.3.1 : Monday, 5 February 2018 18:04:34 UTC