Minutes of the June 4, 2020 Virtual Interim Meeting of the WEBRTC WG

Minutes of the WEBRTC WG Virtual Interim
June 4, 2020
Henrik Bostrom, Scribe

Simulcast Testing Status (Bernard)

We now have simulcast loopback tests in WPT. The tests are based on work done by Fippo, with inspiration from Florent.
The original proof of concept ("Simulcast Playground") used SDP munging to send simulcast and receive it using SSRCs.
These tests could only run on Firefox, but not on other browsers.
The current tests uses SDP munging to send RID header extensions and then receive them as MID header extensions. These
tests now run successfully on Chromium-based browsers as well as (recently) on Firefox.

The test "basic.https.html"  demonstrates the ability to send simulcast and receive and render each encoding, with
variants testing VP8 and H.264/AVC operation. This tests uses addTransceiver to send multiple encodings as well as
the ability to send RID and receive MID header extensions (protocol functionality).

"getStats.https.html" tests that stats corresponding to the RID values sent are retrievable.  It does not test that
the stats are correct.

How can we extend test coverage further?
One direction is to test each video codec. Fippo has created tests for H.264/AVC and VP8 and these have been merged.
This direction could be extended to also test VP9 and other codecs such as AV1.

Another direction is to test RTCRtpEncodingParameters dictionary members.
Fippo has created a test (now merged) that sets active=false and verifies that the encoding is not sent, then sets active=true to turn it on again.
He has also submitted a test (under review) that sets maxFramerate and tests that the framerate sent is less than the maximum.
A similar test could be developed for maxBitrate:  set a value and then test that the bitrate sent is less than the maximum.
Another test would be to set scaleResolutionDownBy and verify that the width and height of the resulting video approximates the requested scaling.

What can't we test in WPT?
Since these are loopback tests, we shouldn't have any loss, so we can't test robustness features such as RRID, RTX, RED or FEC.
Also, since this only tests whether a single browser can receive the streams it sends, we cannot test interoperability with SFUs, MANEs or other browsers.

Discussion

What about KITE testing?
The purpose of these WPT tests is to provide a foundation for interoperability testing, by ensuring that each browser can
receive and render the streams it sends, based on the encoding parameters.  These tests should help us verify basic simulcast
operation and help us ensure that we don't regress.

WPT tests are not a substitute for KITE or E2E testing.  For example, we plan to submit updated KITE tests as part of our effort to bring WebRTC-PC to PR,
to demonstrate protocol interoperability between browsers, MANEs and SFUs.

What about WebRTC-Stats?
So far, we have only tested that WebRTC-Stats are retrievable, but not that they are correct.
Since we can retrieve stats by SSRC and RID, we could use the concepts demonstrated in the simulcast loopback tests to check that
counters on the sender side and receiver side align (using SSRCs and/or RIDs to identify the streams).  This is not perfect, but represents an
improvement on the current state of WebRTC-stats testing.

Followup items

1. WebRTC-Stats testing plan
2. Updated KITE test results to support WebRTC-PC PR effort

Insertable Streams Status (Harald)

Available as an Origin Trial in Chrome M83. Used by Duo and Jitsi for frame encryption.
Also under investigation for other uses.

Spec document now conforms roughly to Chrome M84. Metadata information still needs to be teased out.

Current limitations: No creating frames, can’t manipulate metadata, no interaction with feedback.
These limitations exist because the current API has no mechanism to interact with congestion control.

Extensions envisaged: Support for a feedback path? Bring your own transport? Bring your own encoder?

The Call for Consensus (CfC) concludes on June 5, 2020.  So far the feedback on adoption has been positive.

Eric Rescorla (EKR): Encryption and key management in JavaScript is flawed. Among other things, it circumvents isolated streams.

Harald: Deploying E2E in a different way has other problems. This was needed because the encryption mechanism was not standard.
No encryption versus encryption with key management in JS.

Youenn: I view this as a short-term solution. Long-term we should have E2E provided by the browser. But I think shipping progressively is probably OK.

EKR: Suggests adopting this, but dropping the E2E encryption as an endorsed use case.

Bernard: The only E2E encryption use case remaining in the WebRTC-NV Use Cases document is the "Untrusted Javascript" use case:
https://w3c.github.io/webrtc-nv-use-cases/#untrusted*.  The "Trusted Javascript" use case has already been removed.

Henrik: What other use cases does Insertable Streams relate to?
Bernard: Virtual Reality Gaming (Metadata sent along with the encoded media): https://w3c.github.io/webrtc-nv-use-cases/#vr*
Funny Hats (access to raw media): https://w3c.github.io/webrtc-nv-use-cases/#funnyhats*
Machine Learning (access to raw media): https://w3c.github.io/webrtc-nv-use-cases/#machinelearning*

EKR: Could we have UA supplied transformers for encryption? Seems like a straightforward way forward.

Richard Barnes: That would allow people to experiment with a general API and move over to safer models in the future. Makes sense to have both in parallel.

Harald: Suggest reworking the use cases.

Bernard: The current "Trusted Javascript" use case only refers to the Javascript not being trusted with access to keys.  It doesn't
mention isolated streams.  Do we have a volunteer to help evolve the use cases?

Next steps
EKR to post his thoughts to the mailing list

TCP portscanning (Harald)

Connecting to random TCP ports is a bad idea for security.

Should browser say no or silently not connect?

Suggesting to block specific ports.

Bernard: A fixed "blocked ports list" does not solve all issues. There are many other ports that can be abused.
PBXes are vulnerable to attack on SIP (5060-5070) or Administrative ports (e.g. 8080) which can be used to steal *serious* amounts of money.

Other use cases discussed:

You’re blocking it.

You’re connecting to it and there’s ICE there.

You’re connecting to it and there’s no ICE there.

Maksim: I would prefer an explicit error, because silent inaction could make debugging complex.

Harald: I was inclined toward an explicit error, but perhaps it is better to “Act as if it was a valid candidate that didn’t connect”.

Next steps:
Harald will submit a PR.

Media Capture Automation (Youenn)

Automated testing is hard, but also beneficial. What should be testable?

Calling getUserMedia.

Constraints matching.

devicechange event.

enumerateDevices filtering.

Proposal: Add a WebDriver API.

Control prompt result (accept/reject):

Add/remove fake devices.

Bernard/Jan-Ivar: This is useful.

Henrik: This would have been useful a long time ago. How many tests are needed now that we are almost done? Is this for WPT or for app developers?

Jan-Ivar: When resolving the privacy issues, this will be very useful.

Guido: There are things we have not been able to test.

Harald: Yes, this is needed useful!

Next steps

Youenn to post a proposal to the WEBRTC WG mailing list.


API to request audio output device selection (Youenn)

We have HTMLMediaElement.setSinkID

But the page needs to get device IDs.

Not all pages want camera/microphone. Internet of Things devices (e.g. a speaker) may not have a camera/microphone.

Proposal: prompt user for selection.

Option A: selectAudioOutput() and user selects device in prompt.

Option B: Change setSinkID to prompt if deviceId is null or not in enumerateDevices.

Jan-Ivar: Risk of re-prompt if IDs are persistent. setSinkId risks of re-prompt?

Max: How does the application know if the device changes?

Guidou: I like selectAudioOutput() more because it’s an alternative to getUserMedia(), more compatible.
I see no problem rejecting a deviceId that is not permitted (in enumerateDevices).

Harald: selectAudioOutput() seems like a better name. setSinkID is too cryptic.

Maksim: Why not an API that exposes all devices and let the application pick?

Youenn: There are privacy issues with exposing everything. A device picker is safer from a privacy point of view and is higher in the list of suggestions.

Maksim: An Application can potentially make smarter decisions, for example front camera preferred over back camera in a call application.
Or you could have a microphone and want the same headset.

Youenn: When exposing a device, both headset and microphone could be exposed at the same time.

Setting the audio output for the entire page (Youenn)

Frequently it is desired to have all audio output go to a single device.
Today to accomplish this it is necessary to call setSinkId() on all elements.
This can be done, but it is not "optimizing for the common case".

Proposal: MediaDevices.setSinkId() API to override page default.

Harald: Silly name! Otherwise good.

Jan-Ivar: Is this just a polly-fill?

Youenn: No, because it would also apply to WebAudio and third-party iframe.

WG response: Sounds good.

Deprecate inputDeviceInfo.getCapabilities() for privacy (Jan-Ivar)

Helps applications implement a user picker, but it’s a trove of fingerprinting info!

“user-chooses” provides feature-parity, without the information leak. So can we deprecate it?

Youenn: Safari has not implemented it and is not a fan of it, but does not see it as a privacy issue if the user has already
granted access to the devices.

For the first time in a few months, the WG covered all the slides, and gets to the "Bird" slide.
The Bird in Question (an Anhiga) is named "Sergio".

Meeting adjourned.