Re: Semantics question for onaddstream() callback

On 11/14/2011 11:14 AM, Justin Uberti wrote:
>
>
> On Sun, Nov 13, 2011 at 2:06 PM, Stefan Håkansson
> <stefan.lk.hakansson@ericsson.com
> <mailto:stefan.lk.hakansson@ericsson.com>> wrote:
>
>     On 11/13/2011 05:35 PM, Justin Uberti wrote:
>
>
>
>         On Sun, Nov 13, 2011 at 9:59 AM, Stefan Håkansson
>         <stefan.lk.hakansson@ericsson.__com
>         <mailto:stefan.lk.hakansson@ericsson.com>
>         <mailto:stefan.lk.hakansson@__ericsson.com
>         <mailto:stefan.lk.hakansson@ericsson.com>>> wrote:
>
>             On 11/13/2011 03:33 PM, Justin Uberti wrote:
>
>
>
>                 On Mon, Nov 7, 2011 at 11:51 AM, Stefan Håkansson LK
>         <stefan.lk.hakansson@ericsson.____com
>         <mailto:stefan.lk.hakansson@__ericsson.com
>         <mailto:stefan.lk.hakansson@ericsson.com>>
>         <mailto:stefan.lk.hakansson@
>         <mailto:stefan.lk.hakansson@>____ericsson.com <http://ericsson.com>
>
>         <mailto:stefan.lk.hakansson@__ericsson.com
>         <mailto:stefan.lk.hakansson@ericsson.com>>>> wrote:
>
>
>          >On Mon, Nov 7, 2011 at 7:15 AM, Harald Alvestrand
>         <harald@alvestrand.no <mailto:harald@alvestrand.no>
>         <mailto:harald@alvestrand.no <mailto:harald@alvestrand.no>>
>         <mailto:harald@alvestrand.no <mailto:harald@alvestrand.no>
>         <mailto:harald@alvestrand.no <mailto:harald@alvestrand.no>>>__>
>         wrote:
>          >> 1) We know that this stream is expected. The callback occurs as
>                     soon as we
>          >> have the OFFER or ANSWER containing information about this
>                     stream. Move the
>          >> last onaddstream() up to just after onconnecting().
>          >>
>          >> 2) We know that media is arriving on the stream. The callback
>                     occurs as soon
>          >> as the first media packet arrives. Move the first onaddstream()
>                     down below
>          >> the "ICE Completes" line
>          >>
>          >>
>          >> My order of preference is 1) followed by 2). But I think we need
>                     to pick one
>          >> and only one, and make sure the API spec says exactly that.
>          >
>          >My vote would be (1) and to add a onmediastarted() callback to
>          >indicate the first
>          >packet for a given media stream.
>                     I share this view - you need both. //Stefan
>
>
>                 Agree regarding (1). Regarding (2), what do we expect the
>                 application
>                 would do when it gets an onmediastarted()?
>
>             That's the indication that things worked out. (1) only indicates
>             that signaling has been carried out, (2) tells the
>         application that
>             media is actually getting through. There can be problems with
>             getting media through; there may FWs or NATs or other
>         reasons for
>             not getting it through.
>
>             The application could e.g. wait with displaying the video
>         surface
>             until something is actually received. (Sure there are other
>         ways -
>             paint a canvas&analyze - but (2) is quite convenient)
>
>
>         I agree this is valuable, but typically in this case you want to
>         wait
>         until the first frame is received, not just the first packet;
>         you don't
>         want to start display of video until you actually have something
>         to display.
>
>
>     Yeah. Triggering "onaddstream" once the first frame has arrived
>     might be a better measure than RTP getting through.
>
>     Just for info, we've implemented according to the spec as of August,
>     i.e. fire "onaddstream" when the first RTP packet gets through. We
>     reasoned that if you get one RTP packet the rest of the data needed
>     to display anything will anyway arrive shortly. We've also discussed
>     delaying "onaddstream" until at least one RTP packet has arrived on
>     all tracks belonging to the stream, but concluded that that might
>     delay the event more than you want - you would like to start playing
>     the audio even if the first video data has not arrived. But it would
>     be a safer measure of that all tracks are getting through to the
>     other end.
>
>
> I don't think this is the right direction for onaddstream. It may be
> important for a stream to exist even when no data has yet been received
> on it. Consider the case of an audio conferencing application, which
> wants to indicate that a new participant has joined the conference, but
> no media has been delivered for that participant because they are not
> speaking.

I agree to that there is a need for both events, that's why I said I 
think we should support both (1) and (2) in the original mail. Exactly 
how we do it is up for discussion!

>
> I agree there are many cases where you need to know when media playout
> has begun, but I don't think onaddstream is the right way to signal
> this. You probably want to know on a per-track bases, so some sort of
> state change callback from MediaStreamTrack feels like the best solution
> to me (perhaps the LIVE state can be used for this).
>
>
>
>
>         Failure to receive media also seems like it might be a general
>         problem
>         for the <video/> tag, so perhaps any notification of errors
>         should (or
>         perhaps already do) happen there.
>
>
>             As the API is currently defined, feeding back to the sender
>         that (2)
>             has happened can also be used as an indicator to the sender
>         that the
>             streams sent are received be the receiver (of course we
>         could add
>             error events at the addStream level to accomplish the same
>         thing -
>             but it is already there in some sense with (2)). So it can
>         be part
>             of error handling.
>
>             If I had to select, (2) would be the one to keep; but both
>         make a
>             lot of sense.
>
>             my $0.02
>
>
>
>

Received on Monday, 14 November 2011 10:22:47 UTC