[webrtc-pc] Modify the codec description model to ease describing changes (#2925)

alvestrand has just created a new issue for https://github.com/w3c/webrtc-pc:

== Modify the codec description model to ease describing changes ==
Current discussions on codecs have uncovered several issues with our current model:

* No adequate representation of send-only / receive-only codecs (#2888)
* Hard to describe dynamic revision of the set of codecs as needed for encoded-transform ([encoded-transform #172](https://github.com/w3c/webrtc-encoded-transform/issues/172))
* Lack of clarity in how to get codecs enabled that are not negotiated by default ([webrtc-extensions #100](https://github.com/w3c/webrtc-extensions/issues/100))

This note suggests some changes that may address those issues.

# Current model
As of Jan 11, 2024, the codecs are implicitly described as part of the platform implementation. Their API representation is in the form of the RTCRtpCodec dictionary and its sub-dictionaries RTCRtpCodecCapability (no additional members) and RTCRtpCodecParameters (which adds payload type).

Codecs are manipulated by [setCodecPreferences](https://w3c.github.io/webrtc-pc/#dom-rtcrtptransceiver-setcodecpreferences), and accessed through the [final steps to create an offer](https://w3c.github.io/webrtc-pc/#dfn-final-steps-to-create-an-offer) - where it filters against the list of implemented send codecs and the list of implemented receive codecs (depending on transceiver direction). Codec preferences are stored on transceiver, in the [[PreferredCodecs]] slot.

The final steps to create an offer or answer is not influenced by the current negotiation state, but there is a NOTE in the setCodecPreferences section suggesting that answers should only have codecs that appear in the offer.

# Suggested revised model
Codecs supported, and current preferences, are stored in the sender and receiver, as an ordered list of codecs. Each entry in the list has, in addition to what’s in RTCRtpCodec, two fields:
* A boolean “enabled”
* A payload type that may or may not be present

When creating a sender or receiver, its [[codecs]] slot is initialized to the list of implemented codecs for sending / receiving. Each “enabled” flag is set in a platform dependent manner, and the associated PT is left undefined.

When calling setCodecPreferences, checking is done against receiver.[[codecs]], not against sender/receiver.getCapabilities(). If setCodecPreferences() includes a codec with the “enabled” flag set to false in the receiver’s [[codecs]] slot, it is set to “true”.

When generating an offer or answer, all “enabled” codecs are included in the offer, and the receiver’s PT values are set to the offered values. For sendonly media sections, the initial codecs are taken from the sender; for recvonly media sections, the initial codecs are taken from the receiver; for sendrecv, the union of sender and receiver is used.

When applying a remote offer or answer, the sender’s PT values are set.

If a codec in a remote offer or answer is on the supported list, but “enabled” is false, “enabled” is set to true. This ensures that the codec is included in subsequent offers and answers. (Note: This is the only part of this reimagining that might include a behavior change, but we suspect that this is the way current implementations behave anyway.)

# Possible extensions that are easier in the revised model
These changes need to go into relevant documents until implemented, not in webrtc-pc.
## Adding to the set of supported codecs
This functionality is needed for encoded transform, which may generate payload types that are not natively supported by the platform. The transform acts as a codec. On a receiver transform, attempting to enqueue a frame results in an error. 
This can be done by adding to the sender and receiver’s [[codecs]] list. Potentially this can be done by an API call, or by picking up on input/output codec info on a transform.

## Enabling a disabled codec through API
An API call can be added on sender and on the receiver to get the current codec list including disabled codecs. This can then be put back into setCodecPreferences() which would enable it in offers for recvonly or sendrecv sections. This is nicer if “enabled” is exposed.
## Preventing a codec from being negotiated
Still tough. setCodecPreferences does NOT do this. This is intentional, see [this WPT](https://wpt.fyi/results/webrtc/protocol/additional-codecs.html?label=experimental&label=master&aligned)

## Supporting sendonly codecs
This is easily done by having the codec in the sender’s list only. It will then be offered on sendonly and sendrecv sections. Sendonly codecs are not available to setCodecPreferences().

## Supporting recvonly codecs
As above, but offered only on recvonly and sendrecv sections.

Please view or discuss this issue at https://github.com/w3c/webrtc-pc/issues/2925 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 12 January 2024 09:06:29 UTC