Re: On the way to WebRTC 1.0

> On Oct 19, 2018, at 9:44 AM, Alexandre GOUAILLARD <agouaillard@gmail.com> wrote:
> 
> 
> 
> On Fri, Oct 19, 2018 at 4:20 AM youenn fablet <yfablet@apple.com <mailto:yfablet@apple.com>> wrote:
> Hi,
> 
> I thought a bit on how to best finalize WebRTC 1.0 in terms of specification work.
> I believe we need to at least do two things:
> 1. Resolve all WebRTC 1.0 issues
> 2. Prove (to ourselves and others) that the WebRTC 1.0 spec is implementable in an interoperable manner
> 
> sounds reasonable. 
> 
> For 1, it might be a healthy exercise for the Working Group to label issues as "WebRTC 1.0” vs. “Post WebRTC 1.0”.
> 
> sounds good to me.
>  
> [..]
> 
> For 2, analysis needs to be done on the whole spec to identify holes in our testing coverage.
> I think this effort was done in the past (DrAlex?), maybe there is a need for refreshing it once again?
> 
> We do it at every meeting since the Lisbon meeting in 2016. This year again Soares, who wrote 1000+ of the current 1300+ webrtc tests, will report on the states of WPT. I will report separately on webrtc interoperability.

I hope we can do this analysis once and then carry on test coverage on a PR level.

> 
> Note that WPT calls interoperability the capacity for the same API to behave according to the spec ACROSS browsers (one browser at a time),

This is probably what is needed for WebRTC 1.0 PR which is what we are after in this context.

> while in webrtc interoperability means that two browsers can establish a peerconnection (and some more).It means unified pan vs plan B, it means JSEP, codecs, simulcast, .... We try to raise the potential ambiguity in the past, with no result. We should be very explicit in the scope of webrtc to manage expectations.

Agreed that this is probably what WebRTC implementers and users are after.

> 
> Since the spec will continue evolving, we might also want to take some explicit "test step" for any PR being made so that we can classify each change as:
> - Editorial/No need for test
> - Need for WPT test (and a link to the corresponding WPT test PR)
> - Need for manual test/investigation or KITE test
> 
> While our feeling is that it would be a good thing indeed, our experience with WPT is that a happy few do whatever they want. Review is broken, there is no governance. There is no process in place to enforce that the approach you propose, and it would be likely ignored if it goes in the way of the happy few, or is perceive by them as without merit. 

I think the WebRTC WG is responsible to drive what happens in web-platform-tests/webrtc.
Soares made decisive contributions there. Let’s build on this work.