Re: [webrtc-pc] API for codec performance (#2241)

> The RTCRtpReceiver and RTCRtpSender inferfaces both define a getCapabilities() method, that returns a sequence of RTCRtpCodecCapabilities. Right away we see that RTC users can ask about capabilities on both ends of the wire, not just the local machine. This is something MediaCapabilities can't do.

RTCRtpSender.getCapabilities() lists the codecs and header extensions that can be negotiated for sending, and RTCRtpReceiver.getCapabilities() lists the codecs and header extensions that can be negotiated for receiving. It does not tell you what the other endpoint supports until this is negotiated with SDP (negotiating is performed with createOffer, setLocalDescription, setRemoteDescription and createAnswer, [here's an example](https://codepen.io/anon/pen/JQLEqR?editors=1010)).

Once negotiation is completed, getParameters() will tell you the current settings basically.
RTCRtpReceiver.getParameters() gives you very limited information: it's basically just a list of rids, which doesn't make much sense to me when I think about it, this has to do with simulcast, and while we support sending simulcast, receiving simulcast is kind of outside the spec? (Is this API mature?)
RTCRtpSender.getParameters() tells you about codecs and layers being sent and some properties and control knobs. This is very limited, but lets you know which codec is being sent.

RTCPeerConnection.getStats() also returns information about codecs ([RTCCodecStats](https://w3c.github.io/webrtc-stats/#dom-rtccodecstats)). In a legacy version of the API in Chrome, the [implementation](https://w3c.github.io/webrtc-stats/#dom-rtccodecstats-implementation) string was implemented, and people could use it to figure out if HW or SW codecs was used after the fact, but this was very browser-specific (e.g. "in Chrome, 'libvpx' is used for SW"), and the new getStats() API this has not been implemented (yet), but this is an ugly API surface in my opinion, not designed for interoperability, more like debugging.

If we want to know about stuff like this prior to negotiating, as to allow the application to make wiser decisions, we need "capabilities", not "parameters". getCapabilities() might be one API surface for this, but it is also tied in to RTCPeerConnections and SDP. It would be great if we could de-couple capabilities from signaling protocol.

-- 
GitHub Notification of comment by henbos
Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2241#issuecomment-517222221 using your GitHub account

Received on Thursday, 1 August 2019 10:16:33 UTC