Re: Semantics question for onaddstream() callback

On Sun, Nov 13, 2011 at 2:06 PM, Stefan Håkansson <
stefan.lk.hakansson@ericsson.com> wrote:

> On 11/13/2011 05:35 PM, Justin Uberti wrote:
>
>>
>>
>> On Sun, Nov 13, 2011 at 9:59 AM, Stefan Håkansson
>> <stefan.lk.hakansson@ericsson.**com <stefan.lk.hakansson@ericsson.com>
>> <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>>>
>> wrote:
>>
>>    On 11/13/2011 03:33 PM, Justin Uberti wrote:
>>
>>
>>
>>        On Mon, Nov 7, 2011 at 11:51 AM, Stefan Håkansson LK
>>        <stefan.lk.hakansson@ericsson.**__com
>>        <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>
>> >
>>        <mailto:stefan.lk.hakansson@__**ericsson.com <http://ericsson.com>
>>
>>        <mailto:stefan.lk.hakansson@**ericsson.com<stefan.lk.hakansson@ericsson.com>>>>
>> wrote:
>>
>>
>>         >On Mon, Nov 7, 2011 at 7:15 AM, Harald Alvestrand
>>        <harald@alvestrand.no <mailto:harald@alvestrand.no>
>>        <mailto:harald@alvestrand.no <mailto:harald@alvestrand.no>>**>
>> wrote:
>>         >> 1) We know that this stream is expected. The callback occurs as
>>            soon as we
>>         >> have the OFFER or ANSWER containing information about this
>>            stream. Move the
>>         >> last onaddstream() up to just after onconnecting().
>>         >>
>>         >> 2) We know that media is arriving on the stream. The callback
>>            occurs as soon
>>         >> as the first media packet arrives. Move the first onaddstream()
>>            down below
>>         >> the "ICE Completes" line
>>         >>
>>         >>
>>         >> My order of preference is 1) followed by 2). But I think we
>> need
>>            to pick one
>>         >> and only one, and make sure the API spec says exactly that.
>>         >
>>         >My vote would be (1) and to add a onmediastarted() callback to
>>         >indicate the first
>>         >packet for a given media stream.
>>            I share this view - you need both. //Stefan
>>
>>
>>        Agree regarding (1). Regarding (2), what do we expect the
>>        application
>>        would do when it gets an onmediastarted()?
>>
>>    That's the indication that things worked out. (1) only indicates
>>    that signaling has been carried out, (2) tells the application that
>>    media is actually getting through. There can be problems with
>>    getting media through; there may FWs or NATs or other reasons for
>>    not getting it through.
>>
>>    The application could e.g. wait with displaying the video surface
>>    until something is actually received. (Sure there are other ways -
>>    paint a canvas&analyze - but (2) is quite convenient)
>>
>>
>> I agree this is valuable, but typically in this case you want to wait
>> until the first frame is received, not just the first packet; you don't
>> want to start display of video until you actually have something to
>> display.
>>
>
> Yeah. Triggering "onaddstream" once the first frame has arrived might be a
> better measure than RTP getting through.
>
> Just for info, we've implemented according to the spec as of August, i.e.
> fire "onaddstream" when the first RTP packet gets through. We reasoned that
> if you get one RTP packet the rest of the data needed to display anything
> will anyway arrive shortly. We've also discussed delaying "onaddstream"
> until at least one RTP packet has arrived on all tracks belonging to the
> stream, but concluded that that might delay the event more than you want -
> you would like to start playing the audio even if the first video data has
> not arrived. But it would be a safer measure of that all tracks are getting
> through to the other end.


I don't think this is the right direction for onaddstream. It may be
important for a stream to exist even when no data has yet been received on
it. Consider the case of an audio conferencing application, which wants to
indicate that a new participant has joined the conference, but no media has
been delivered for that participant because they are not speaking.

I agree there are many cases where you need to know when media playout has
begun, but I don't think onaddstream is the right way to signal this. You
probably want to know on a per-track bases, so some sort of state change
callback from MediaStreamTrack feels like the best solution to me (perhaps
the LIVE state can be used for this).

>
>
>
>> Failure to receive media also seems like it might be a general problem
>> for the <video/> tag, so perhaps any notification of errors should (or
>> perhaps already do) happen there.
>>
>>
>>    As the API is currently defined, feeding back to the sender that (2)
>>    has happened can also be used as an indicator to the sender that the
>>    streams sent are received be the receiver (of course we could add
>>    error events at the addStream level to accomplish the same thing -
>>    but it is already there in some sense with (2)). So it can be part
>>    of error handling.
>>
>>    If I had to select, (2) would be the one to keep; but both make a
>>    lot of sense.
>>
>>    my $0.02
>>
>>
>>
>