[webrtc-extensions] Suggest API for generating a Stream rather than a MediaStreamTrack from incoming media (#85)

alvestrand has just created a new issue for https://github.com/w3c/webrtc-extensions:

== Suggest API for generating a Stream rather than a MediaStreamTrack from incoming media ==
There's been expressed a desire to detach the codec functionality from the transport handling functionality of a PeerConnection.
This is part of the gradual process of componentizing the PeerConnection, but suggests a different slice than what had been proposed in [[ORTC]].

Suggestion:

- New event "onmedia", which contains a Stream object, as well as metadata (like media type, PT and so on) that allows identifying what the PeerConnection knows about how the incoming Stream's frame should be decoded
- If the Stream object is unlocked (has no Reader) when the event handler returns, or there is no event handler, the PeerConnection will instantiate a decoder, will create a MediaStreamTrack with the output of the decoder, and fire an "ontrack" event.
- If the Stream object is locked when the event handler returns, the PeerConnection will not instantiate a decoder, will not create a track, and will not fire an "ontrack" event. Instead, it will push EncodedAudioFrames or EncodedVideoFrames as appropriate onto the Stream object; these will be pushed as soon as they are assembled, without regard to RTP sequence numbers. The sequence numbers will be available as metadata.

Input sought on whether this seems to make sense, or if it has some glaring deficencies compared to present APIs.

Please view or discuss this issue at https://github.com/w3c/webrtc-extensions/issues/85 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 13 October 2021 16:18:42 UTC