Re: Issue 141: Mandatory "kind" in RTCRtpReceiver constructor

On Tue, Aug 5, 2014 at 10:10 AM, Shijun Sun <shijuns@microsoft.com> wrote:
> Apps need to create the rendering pipeline by (a) creating the medisStream object with the receiver.track, and (b) setting it on video.srcObject.

And they can do that after calling RtpReceiver.receive().  Why would
they need to do so before calling receive()?  If there is a good
reason to have a track ready to go before calling receive() then we
would indeed need to pass in the kind to the RtpReceiver.  But I
haven't seen the use case for it yet.

>
> BTW, Step (a) is not needed for RTCPeerConnection.
>
> If the UA doesn't know the 'kind' until data arrives, the receiver will probably have to fire an event to let the app know that it is time to build the rendering pipeline and will have to buffer up the streams internally before the pipeline is ready.

The UA knows as soon as receive() is called, which is before it will
process any media.

>
> The challenges are:
> 1. we don't have the event defined yet to tell app when the data actually are received
> 2. it will cause some unnecessary startup latency when building rendering pipeline after data received
> 3. how much streams/samples to buffer within the receiver can be tricky since some of the JS operations will have to be on the UI thread
>
> Having the 'kind' of the receiver in the constructor as normative can get around these problems, so apps can start building the rendering pipeline right after the receiver object is created.
>
> -Shijun
>
> -----Original Message-----
> From: Peter Thatcher [mailto:pthatcher@google.com]
> Sent: Tuesday, August 5, 2014 9:52 AM
> To: Bernard Aboba
> Cc: public-ortc@w3.org
> Subject: Re: Issue 141: Mandatory "kind" in RTCRtpReceiver constructor
>
> Wouldn't the RtpReceiver know if it's audio or video as soon as the RtpParameters are passed into receive()?  If you have audio codecs, you are audio.  If you have video codecs, you are video.  If you have
> neither, error out.  If you have both, error out.   And previous to
> calling receive(), I think .track should be null.  I think it's OK for it to be nullable even if the WebRTC 1.0 is not.
>
> On Mon, Aug 4, 2014 at 9:17 PM, Bernard Aboba <Bernard.Aboba@microsoft.com> wrote:
>> Should the "kind" argument be mandatory (not optional) in RTCRtpReceiver?
>>
>> If done, the RTCRtpReceiver interface would look like this:
>>
>> [Constructor(DOMString kind, RTCDtlsTransport transport, optional
>> RTCDtlsTransport rtcpTransport)] partial interface RTCRtpReceiver :
>> RTCStatsProvider { readonly attribute MediaStreamTrack track; };
>>
>> Not that "readonly attribute MediaStreamTrack track" is also used in the WebRTC 1.0 RTCRtpReceiver definition:
>> https://www.w3.org/2011/04/webrtc/wiki/images/6/63/WebRTC_RTCSender-Re
>> ceiver.pdf
>>
>> Related Issue #139:
>>
>> In ORTC API, the "kind" of RTCRtpReceiver.track is unset before packets are received. While we have RTCRtpCodecCapability.kind, there is no equivalent attribute within RTCRtpParameters (or RTCRtpCodecParameters). As a result, when RTCRtpReceiver.receive(RTCRtpParameters) is called, the browser cannot deduce the "kind" of track it will be dealing with, which results in delays in setting up the audio or video pipeline.
>>
>> Does it make sense to pass the "kind" in the RTCRtpReceiver constructor?
>>
>> [Constructor(RTCDtlsTransport transport, optional RTCDtlsTransport
>> rtcpTransport, optional DOMString kind)]
>>
>> Looking at the proposal for integration of RTCRtpReceiver in WebRTC 1.0, it looks like this problem does not arise there:
>> https://www.w3.org/2011/04/webrtc/wiki/images/6/63/WebRTC_RTCSender-Re
>> ceiver.pdf
>>
>> In WebRTC 1.0 RTCRtpReceiver.track is not nullable, so RTCRtpReceiver.track.kind is presumably available.
>> RTCRtpReceiver is vended when a remote track is added. Since this happens when the sender adds a track (e.g. sender = pc.addTrack(mst);), the "kind" of the added track is set on both the sender and the receiver.
>

Received on Tuesday, 5 August 2014 17:48:41 UTC