Minutes of the July 8, 2020 WEBRTC WG Virtual Interim

WebRTC Virtual Interim Minutes
July 8, 2020
Henrik Bostrom, Scribe

Mixed Codec Simulcast (Florent)

Today the WebRTC 1.0 API does not support using different codecs for different simulcast layers.
Some codecs may be better for certain resolutions/bitrates. Examples:

Low resolution AV1, higher resolution VP8/VP9.
HW codecs for some layers, SW codecs for some layers.

Originally, this feature was supported in ORTC as well as in previous versions of the WebRTC 1.0 API.  It had to be removed from WebRTC 1.0 because codecs were referenced by payloadType, which was not known prior to O/A negotiation.

Proposal: Instead of referencing codecs by payload type, use mime type instead, since this can be obtained from getCapabilities().  As with setCodecPreferences() we can filter the list and set up multi-codec simulcast using addTransceiver() or setParameters(). This would be added to webrtc-extensions to allow experimentaton to see if it is useful.

Discussion: Why not use multiple senders instead?
Florent: A single sender is preferred over multiple senders because with a single sender you have unified congestion control, allowing layers to be degraded in a specific order.  There is no priority mechanism defined between multiple senders.

Tim: What happens on the wire?
Florent: You negotiate the payloadType using Offer/Answer.
Bernard: I guess you wouldn’t want to change the clockRate?
Florent: You wouldn’t be able to, it has to match getCapabilities().

Henrik: This would look the same as changing the codec mid call which can be achieved with an “offer-answer dance”. I think this would even be compatible with a client that does not support this API, because when negotiating you ensure that you are prepared to receive it.

Conclusion: There is interest in it. Florent to submit a PR to webrtc-extensions.

Browser/SFU Capability Negotiation (Dr. Alex/Bernard)

AV1 can support multiple encodings on a single RTP stream, but in SDP you can only negotiate multi-stream simulcast. So multiple encodings on a single RTP stream is not negotiated in Offer/Answer, it is turned on in setParameters() or addTransceiver().

In the process of developing tests for support of AV1 in WebRTC, questions arose because it is possible to support "encodings within encodings": several streams, each containing multiple encodings. Does Dr. Alex have to write tests for this?

Question: Should “mixed” simulcast transport be allowed?

Henrik: What is the benefit?
Bernard: We couldn’t think of any benefits.
Florent: If it can be negotiated, why not let people try if they want to experiment?
Dr. Alex: It’s better for an SFU to know what it must be prepared to support.  If we don't have a use case, it's better to reject it.
Henrik: If we don’t have a use case we should forbid it.
Dom: If it is later needed we can revisit.

Conclusion: Accept the proposed PR to WebRTC-SVC, throw an exception for now.

WebRTC-ICE Status Report (Bernard)

Standalone ICE transports have been implemented in Chrome, Edge, Edge Spartan, ortclib.
Currently only ortclib supports forking, the browser implementations do not.  However, the current browser implementations do not meet the requirements for the following use cases:

Multi-party online games (requires forking)
Calling multiple endpoints (requires forking)
Mobility (requires FlexICE)
P2P mesh use case (requires FlexICE and forking).

So the problem is that we have a WebRTC-ICE specification which does not fully support any use case in the WebRTC-NV Use Cases document.

FlexICE features:
Forking, cell/wifi control, checking activity/frequency control, continual gathering...

To give us a perspective on this, we have invited a distinguished guest (Peter Thatcher) who is an author of WebRTC-ICE,  and is now an application developer at Signal, which uses ICE forking.  Since Peter has been a browser developer as well as an application developer, he has a unique perspective.

Peter Thatcher on ICE/RTCPeerConnection forking

From TPAC 2019 I ended with these questions: Are we willing to do ICE forking? Are we willing to implement ICE forking? How hard is it to implement?

I have since tried, and it turns out, it’s not that hard!

If we know that it is possible to implement, and not that hard. The question that remains is: Why do we want this?

P2P meshes
Call Forking: One device calling multiple devices, ringing them simultaneously.

Why do you need ICE forking for call forking?
You can do serial forking, calling one endpoint at a time, then moving to the next one. But this makes call setup slow...

Why not send out N offers?
You need to know N in advance. More candidates to allocate and signal. Everything multiples on top of N. Inefficient.

Can we fork RTCPeerConnection? Turns out: Yes we can!

This puts the following requirements on the API:

Creating offers that can be shared.
Gathering local ICE candidates that can be shared.
Applying those shared things on a newly created RTCPeerConnection.

Experimenting on a native app shows this is all possible. But is there interest?

Discussion:

Question: What is the difference between forking and a new PC?
Answer: No need to gather ICE candidates again.

Are the ports shared? Yes.
Are the transports shared? No. IceTransport and DtlsTransport objects are distinct.

Conclusion:
1. Add the call forking use case to WebRTC-NV.
2. Develop a PR. To be decided if it should be for WebRTC-ICE or WebRTC-Extensions.

WebRTC-NV Use Cases (Bernard)

Currently, the WebRTC-NV Use Cases document includes use cases in two categories:
1. Improvements to existing use cases
2. New use cases

Additional use cases that have been suggested:
Secure conferencing
  Trusted JavaScript use case (removed from the document).
  Semi-trusted Javascript. Needs definition.
Data channel in workers.
P2P mesh use cases.
Broadcasting 1:Many.
Censorship circumvention/VPN.
More control over latency/acceptable loss.

A W3C WebTransport WG is being formed that will develop its own use case document.  Some of the WebTransport use cases suggested at IETF 106 are:

New use cases:
Machine learning (client/server).
Cloud gaming.
Live streaming.

Existing use cases:
Remote virtual desktop.
Web games.
Web chat.

Broadcasting 1:Many

Issues today implementing this with WebRTC:
Time spent encrypting/decrypting is high, can’t prefer HW.
ICE over TCP not being ubiquitous
Lack of DRM.

Bernard: The issue with negotiation of more efficient ciphersuites is solvable in WebRTC, but the others are easier to address in WebTransport, which is client/server (so no need for ICE) and can handle reliable transport as well as DRM (by transporting containerized media).

Proposal: Transfer to WebTransport use cases.

Conclusion: No objection.

Censorship circumvention/VPN

Today, VPNs are being constructed over the WebRTC data channel to address fingerprinting concerns, such as:
Information from ICE (ufrag/pwd).
DTLS handshake (certificate).

Bernard: DTLS 1.3 can shield the certificate from exposure on the wire. But as long as WebRTC uses ICE, we will have issues with the ufrag/pwd. If a VPN is being contemplated, is this more of a client/server use case, which could be handled via a QUIC VPN, with no ICE?

Proposal: Transfer to WebTransport use cases.

Tim: I think this is an interesting use case, it may be worth having in WebRTC. It’s the way to do peer-to-peer today.
Bernard: We could look at privacy oriented use cases in WebRTC. Or have VPN as a separate WebTransport use case.

Conclusion: Try to keep and develop this in WebRTC-NV use cases and privacy could be of interest to WebTransport as well, so perhaps it could go to both places.

More control over latency/acceptable loss

IoT/Security camera developers are asking for:
Configurable upper/lower bounds on latency.
Lossless mode.

Bernard: WebRTC can allow you to configure rtx-time, so that media won't be retransmitted too long. But WebTransport can support reliable transport as well as datagrams
Tim Panton: QuicTransport is simpler to implement on an IoT device than HTTP/3.

Action Item: Refer this to WebTransport but see if there a also an IoT use case relating to WebRTC that belongs be in WebRTC-NV.

Have we missed any use cases? (Bernard)

Example use cases since the Pandemic started:

Industry Conventions online.
Art showings online.
Virtual windows with streaming video.
Political conventions.
Nationwide town hall (100K+ participants)
Virtual movie theatres.
Virtual sporting events.
Virtual performances with custom backgrounds.
TV shows shot at home:
  SNL at home.
  “Full Frontal” at home.
Virtual concerts.
Virtual travel. WebVR?

Henrik: These use cases impact scale a lot and possibly interactiveness.
Dr. Alex: You may need the capacity for anybody in the audience to raise their hands, but not all of them at the same time.
Tim: A lot of crossover between WebRTC and WebTransport. It would be nice if there was a clear way to cross the line between them.
Dr. Alex: Sharing the same stack seems to be a big ask.
Bernard: Is it sharing the same stack or the ability to mix video on the endpoints?
Florent: There’s an assumption if you have good latency you should have good quality,
Dr. Alex: Synchronized delivery seems to be the weakest link. You can’t speed everyone up, so you have to slow everyone down to the slowest link. For sports, you want to have faster delivery.
Florent: Synchronized playing between multiple participants could be of interest.
Bernard: You should be able to do both WebTransport and WebRTC in the same application with the desired synchronization (and record the end result).
Dr. Alex: Business model: some people have low value content, some have high value content. Do you have DRM or end-to-end encryption? There may be value in standardizing this.
Bernard: DRM will inevitably come up in the uses involving commercial content (e.g. sporting events, TV shows)
Tim: What about protected streams?
Bernard: You’d need to combine it with encryption.

Name that bird!
Bird contributed by Stefan Hakansson.