Re: "onaddstream" and beyond

On Fri, Nov 18, 2011 at 4:40 AM, Stefan Håkansson LK <
stefan.lk.hakansson@ericsson.com> wrote:

> Group,
>
> recently there has been some discussion (the starting point was <
> http://lists.w3.org/Archives/**Public/public-webrtc/2011Nov/**0015.html<http://lists.w3.org/Archives/Public/public-webrtc/2011Nov/0015.html>>)
> on when the "addstream" event should fire (and a MediaStream object should
> be handed over to the application):
>
> a) Should it be when the signaling setting up the associated RTP-streams
> over the PeerConnection happens?
>
> or
>
> b) Should it be when there is some data arriving (in at least one track)
> for the MediaStream?
>
> I think we've have concluded that there is a need for the application to
> know when an incoming MediaStream has any activity in it. This is automatic
> in the b) solution as the MediaStream object will not be available before
> there is any content. But with the a) model, there could be an attribute,
> or event fired (possibly on the track level), in the MediaStream object
> that informs the application on when there is any media actually streaming.
>
> So basically there are two options:
>
> 1) Only create MediaStream objects when there is any media streaming
>
> 2) Create MediaStream objects immediately, and have properties of the
> MediaStream tell the application whether or not any media is streaming
>
> Currently in the spec, model 1) is used. This is valid both for
> getUserMedia (which operates asynchronously) and when an incoming stream is
> being set up in the PeerConnection.
>
> The question to this group is: should we stay with this model, or should
> we move to model 2)? I think regardless of model selected, it should be
> applied to both getUserMedia (i.e., if model 2 is selected, a MediaStream
> object is delivered immediately, and the application can detect when there
> is any media streaming in it) and PeerConnection creating MediaStreams.
>
> I think we need to select one model! What are your preferences?
> (Personally my preference would be to stay with 1) as it is to me
> conceptually simpler and that there are fewer things to add/change in the
> spec -  I worry about our schedule :-) ).
>
>
Thanks for laying out the details here. I agree we should have a single
model, but my preference is #2. Here are my reasons:
1) For incoming streams, if the notification of a stream is not provided
until the first frame/bytes arrive, the application may not be able to hook
up a renderer in time to play out the first samples of media, causing media
clipping.
2) For outgoing streams, devices often take finite time to start. If we
don't get a callback indicating the existence of a stream right away, this
may cause call setup to take longer than necessary.
3) Streams may contain multiple tracks. It's not clear what the behavior
would be when media arrives for one track but not the other; if we fire the
notification, this will be incorrect for at least one track; of we don't
fire the notification, media clipping will occur.
4) Some applications may use onaddstream as notification that a participant
or a source of a participant has been added, and want to display this new
participant/source in the application UI. In conference scenarios, media
may not be received from non-speaking sources for long periods of time,
causing the UI to be out of sync.
5) If we use onaddstream to indicate that a stream exists, and another
callback to indicate that media has been received, an application can still
choose to wait until media is received to act. The app does not have this
choice with a single callback.
6) To me, signaling onaddstream when a stream is created, and
onremovestream when a stream is destroyed, is more consistent and
conceptually simpler than breaking this symmetry and signaling onaddstream
only when media is received.

Regards,
--justin