Minutes of the February 2021 WEBRTC WG Virtual Interim Meeting

Minutes of the WEBRTC WG Virtual Interim
February 22, 2021
Scribe: Henrik Bostrom

Test coverage (Bernard Aboba)

Some limitations of current WPT tests:
Stats: only test presence, not that the values are correct.
Priority, DSCP, content hints: Only test attribute presence, not functionality.
Extensions: Not testing if RTP header extensions are delivered.
Etc. See slides.

AV1 integration testing found many issues in the stack (not just AV1).
But can we run it on a continuous basis to validate PRs? No.

How do we ensure features don’t regress? Should we add server support to WPT?

Proposal (Harald and Philipp)

Can we make incremental progress? Extensive server logic is hard to maintain, so we need to keep it simple.

Proposal: Content reflector.

A server written using the aiortc python module is only 60 lines of code, terminates STUN + DTLS, decrypts SRTP, echoes SRTP/RTCP packets via WebSockets.

WebTransport WPT tests also use a server, based on aioquic.

Discussion:

Do we want to land this? What can we test?

Most useful to test “what actually gets to the server”. E.g. verify if a key frame is sent.
An example: We can test whether an RTCP BYE is sent when it shouldn't be. This represented a bug that was fixed but has regressed off and on.

Can an aiortc server receive simulcast? May need similar tricks as were used in simulcast playground.

Where does the server logic live? This is a python-based server reflecting traffic.

Could we test SCTP? Yes but we would need a JS parser for SCTP.

Other things the current prototype doesn't do that we might want:
1. Ability to generate SDP on the server (so it can send an Offer for the browser to respond to). aiortc also supports ORTC, so it
is possible to reduce the server to an ORTC IceTransport + DtlsTransport.

2. Ability to send RTCP packets from the test server that the browser will react to.
Example: Test server sends a PLI, expects a key frame within X ms.

3. Ability for the test server to generate RTCP dynamically.
Example: Ability to send REMB messages, so the browser won't limit bandwidth to 300 Kbps.

Cullen: What if we used a full webrtc stack instead but added a packet filter? Would be more work on the server side,
but could write more powerful tests.

Decision:

Work on landing a server that can be used with WPT.
Look at addressing some of the limitations of the current prototype.

Talk to WPT folks to understand any constraints they have on the server.

Media capture: Origin Isolation (Jan-Ivar)

Jan-Ivar explains tainted canvas. Cross-origin videos can be displayed on canvas but pixels cannot be read because they are “tainted”.

But today, if you attempt video.captureStream(), the track is zero content according to spec.

Firefox “taints” the MediaStreamTrack, but still allows locally displaying it.

Raw media access is limited to same-origin media. May be more performant, but no urgent use cases. How should we close the issue?

Proposal A: “If the user agent supports tainting, then…”

Proposal B: Band-aid in mediacapture-fromelement, “tainted tracks can only be consumed in HTMLMediaElement sinks…”

Proposal C: Say nothing.

Discussion:

Youenn: We don’t want to add MUST statements to mediacapture-main at this point in the process (headed to PR).

Henrik: Could we just add a placeholder internal slot?

Jan-Ivar: We’re just trying to avoid saying we cannot have tainted tracks.

Youenn: Currently it is not described, so I would keep it that way. We do not currently have consensus that we want to build on tainted tracks.

Decision:

Move this to the media capture extensions spec.

Invalid TURN credentials (Henrik)

We don’t specify where to surface an error if TURN credentials are invalid, but we do have pc.onicecandidateerror with errorCode.
The IANA registry of existing STUN/TURN errorCode values is here: http://www.iana.org/assignments/stun-parameters/stun-parameters.xhtml

Example: RFC 8656 (TURN) defines error code 441 "Wrong Credentials".

The WebRTC specification defines a new error code 701 (outside the IANA range) used when the ICE server is unreachable.

Proposal: use defined errorCode values for non-parse TURN failures.

Discussion: "makes sense"

Conclusion: Make a PR.

Enable opus stereo (Henrik)

“stereo=1” says you prefer to receive stereo, “stereo=0” which is default says you prefer to receive mono. We currently do mono, but Chromium has SDP munging to enable sending stereo based on “stereo=1” (what you prefer to receive) which is backwards.

Proposal: Make “stereo=1” the default, send min(track’s channelCount, stereo attribute) instead. This works with replaceTrack(). gUM() and WebAudio can control channelCount of tracks.

Discussion:

Cullen: What do you intend to do with the offer stereo attribute?

Henrik: I was unaware that existed, but it should not make a big difference because you have to be prepared to receive it regardless and I’m trying to avoid another API for controlling it.

Harald: How would you control what stereo attribute is in the answer SDP? You may need to look at RTCRtpReceiver.track.applyConstraints() to specify channel count for the receiving track.

Henrik: I really want to avoid applyConstraints() modifying negotiation, in that case we would need to also fire onnegotiationneeded. Besides this is only information to the other endpoint what they should send you, it has no effect on the local side (since you have to be prepared to receive regardless), so there is no need to SDP munge at the receiver until after setLocalDescription().

Jan-Ivar: I share those concerns. The simple solution would be to have the sender decide.

Henrik: I don’t think so, because you need a stereo source, so probably that would be rare. But I haven’t checked.

Youenn/Jan-Ivar: What about backwards compatibility? Would this cause us to start sending stereo all the time

Harald: I'm not sure how you would be able to know if there are backwards compatibility issues until hitting stable...


Conclusion:

The proposal seems fine unless this causes compatibility problems. Henrik to dig in more about compatibility issues: see if there are channelCount use counters, check defaults. If things look good consider making a PR.


Transferable data channels (Youenn)

Data channels are used for data being processed in workers.

Proposal: Make data channels transferable.

No solution to persistent data channels, cannot share channels between workers, but reduced problem space.

Work needed: Transfer algorithm, define neutered data channel. Looks relatively easy to allow patch data channels to skip the main thread. Alternative solutions: apply WebSocketStream?

Is there interest in defining and implementing transferable data channels?

Discussion:

Bernard: I think there are several applications for this.

Jan-Ivar: I think we should do one or the other. Both alternatives sound appealing.

Youenn: Happy to discuss either, but think transferring is easier.

Bernard: I like the easier proposal.

Harald: I see implementation hardship. You’ll have two objects being connected to two separate objects - the peer connection and the SCTP data channel.
This will result in an extra copy.

Bernard: Would you have to move both the RTCDataChannel and the SctpTransport objects?

Youenn: We should only move the data channel. And start with the dedicated worker. I could prototype.

Jan-Ivar: What is the status of WebSocketStream?

Harald: WebSocketStream’s Origin Trial ended in M87 and I don’t see any revival in later versions.

Harald: I like the proposal if there aren’t any implementation hurdles.

Youenn: We should at least try.

Jan-Ivar: Blocking on WebSocketStream could drag out specification and may be more difficult to implement. It’s worth exploring transferring the channel as-is.

Conclusion:

There is interest. It would be useful to prototype at least.

Expose RTC transforms in workers (Youenn)

How to expose RTCRtpScriptTransform counterpart in worker?

Option A: Event-based variant. Pros: small API surface.

RTCTransformEventData { readable, writable, options }

Option B: Callback-based variant. Pros: easy to expose additional API surface.

Extend RTCScriptTransform base interface with the attributes on it.

Proposal: Option A.

Discussion:

Jan-Ivar: You can’t have a dictionary as an attribute in WebIDL.

Harald: I like the first option because it is very similar to the current examples. And if you can have a standardized message for the feedback that would make life easier.

Youenn: So we would be fine with an RTC transform event?

Jan-Ivar: When does the event fire and how many times?

Youenn: Whenever you create a new RTCRtpScriptTransform you provide a worker, so that’s when you would queue the event to fire the event in the worker’s event loop.

Jan-Ivar: OK, sounds reasonable, unless there is a common way to do this already.

Youenn: I prefer it this way because it is lighter than the alternatives.

Conclusion:

Write a PR and keep prototyping.

Investigate if this needs to be a dictionary or interface.