Re: Callback on track added? (Re: Semantics question for onaddstream() callback)

On Tue, Nov 15, 2011 at 10:23 AM, Stefan Håkansson LK <
stefan.lk.hakansson@ericsson.com> wrote:

> On 11/14/2011 12:50 AM, Harald Alvestrand wrote:
>
>> (top-posting, apologies - but we're now perhaps discussing overarching
>> architecture....)
>>
>> It seems to me that onaddstream() is logically a MediaStream level event
>> while onmediastarted() is logically a MediaStreamTrack level event.
>>
>> Perhaps the correct interface is to define an onmediastarted() event on
>> the MediaStream, and have that be triggered when the underlying
>> implementation has been convinced that enough media has arrived to do
>> something with? (Yes, leaving wiggle room to do both first-frame and to
>> do first-packet).
>>
>
> Somehow it is not totally clear to me yet what the basic problem of the
> current "onaddstream" is. The usual behavior would presumably be to attach
> a MediaStream it to an audio or a video element for playout, and having it
> fire once there is data coming in at least one track seems appropriate to
> me. If it is when there is one frame or one RTP packet received is another
> story.
>

Just because no data is arriving doesn't mean the stream doesn't exist
(consider silence suppression). I think the app will want to be able to
hook up the MediaStream to a playout element as soon as it exists.

>
>
>> At the moment I don't think we have an event that can be watched for to
>> catch streams as they are defined, and in Santa Clara, we decided that
>> the tracklist is NOT immutable - which means that we have to define how
>> people are supposed to watch it for changes, too.
>>
>>                           Harald
>>
>> On 11/14/2011 12:35 AM, Justin Uberti wrote:
>>
>>>
>>>
>>> On Sun, Nov 13, 2011 at 9:59 AM, Stefan Håkansson
>>> <stefan.lk.hakansson@ericsson.**com <stefan.lk.hakansson@ericsson.com>
>>> <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>>>
>>> wrote:
>>>
>>>    On 11/13/2011 03:33 PM, Justin Uberti wrote:
>>>
>>>
>>>
>>>        On Mon, Nov 7, 2011 at 11:51 AM, Stefan Håkansson LK
>>>        <stefan.lk.hakansson@ericsson.**com<stefan.lk.hakansson@ericsson.com>
>>>        <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>
>>> >
>>>        <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>
>>>
>>>        <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>>>>
>>> wrote:
>>>
>>>
>>>        >On Mon, Nov 7, 2011 at 7:15 AM, Harald Alvestrand
>>>        <harald@alvestrand.no <mailto:harald@alvestrand.no>
>>>        <mailto:harald@alvestrand.no <mailto:harald@alvestrand.no>>**>
>>>
>>>        wrote:
>>>        >> 1) We know that this stream is expected. The callback occurs as
>>>           soon as we
>>>        >> have the OFFER or ANSWER containing information about this
>>>           stream. Move the
>>>        >> last onaddstream() up to just after onconnecting().
>>>        >>
>>>        >> 2) We know that media is arriving on the stream. The callback
>>>           occurs as soon
>>>        >> as the first media packet arrives. Move the first onaddstream()
>>>           down below
>>>        >> the "ICE Completes" line
>>>        >>
>>>        >>
>>>        >> My order of preference is 1) followed by 2). But I think we
>>>        need
>>>           to pick one
>>>        >> and only one, and make sure the API spec says exactly that.
>>>        >
>>>        >My vote would be (1) and to add a onmediastarted() callback to
>>>        >indicate the first
>>>        >packet for a given media stream.
>>>           I share this view - you need both. //Stefan
>>>
>>>
>>>        Agree regarding (1). Regarding (2), what do we expect the
>>>        application
>>>        would do when it gets an onmediastarted()?
>>>
>>>    That's the indication that things worked out. (1) only indicates
>>>    that signaling has been carried out, (2) tells the application
>>>    that media is actually getting through. There can be problems with
>>>    getting media through; there may FWs or NATs or other reasons for
>>>    not getting it through.
>>>
>>>    The application could e.g. wait with displaying the video surface
>>>    until something is actually received. (Sure there are other ways -
>>>    paint a canvas&analyze - but (2) is quite convenient)
>>>
>>>
>>> I agree this is valuable, but typically in this case you want to wait
>>> until the first frame is received, not just the first packet; you
>>> don't want to start display of video until you actually have something
>>> to display.
>>>
>>> Failure to receive media also seems like it might be a general problem
>>> for the <video/> tag, so perhaps any notification of errors should (or
>>> perhaps already do) happen there.
>>>
>>>
>>>    As the API is currently defined, feeding back to the sender that
>>>    (2) has happened can also be used as an indicator to the sender
>>>    that the streams sent are received be the receiver (of course we
>>>    could add error events at the addStream level to accomplish the
>>>    same thing - but it is already there in some sense with (2)). So
>>>    it can be part of error handling.
>>>
>>>    If I had to select, (2) would be the one to keep; but both make a
>>>    lot of sense.
>>>
>>>    my $0.02
>>>
>>>
>>>
>>
>