Re: Handling simulcast

On 6 September 2013 02:17, Harald Alvestrand <harald@alvestrand.no> wrote:
> If we want to allow simulcast to be implemented at application level, it
> seems to me that signalling which tracks should be disabled at the relay is
> also an application level issue, and doesn't need standardization.
>
> As long as the communicating participants have the identifiers they need to
> identify the tracks and streams involved (<msid...>), they can send metadata
> outside of any standardized interfaces.

Absolutely agree.

> I'm not sure srcname helps at all. We already know that the video tracks are
> in the same stream, and that both are enabled. If simulcast is implemented
> as an application-layer function, the user agent can have the interpretation
> that these are versions of the same stream.

SRCNAME (or something like it) is necessary if you intend to have
seamless playback.  Otherwise, the <video> tag can't tell if these are
the same scene, or two different scenes.  Besides, when the sender
sets bigTrack.disabled=true, the playback will show black, as opposed
to switching to the backup.  Actually, maybe sending black is a bad
idea when there is a simulcast track present.

> The completely manual, JS-driven approach is to wait for the "error" event
> described under "if the media data is corrupted", use that to disable the
> current video stream, and enable a backup stream.

That's going to be OK for some classes of use, but it's not going to
be anywhere near responsive enough to deal with cases where the big
track drops out.

> What we could do, hypothetically, to automate this is to suggest that the
> HTML5 media load algorithm be changed - add a step on video playback that
> says something like:
>
> "If multiple video streams are present in the resource, and the currently
> selected video stream does not provide data that allows a picture to be
> rendered at that time, the user agent may switch to the next enabled video
> stream in the resource for the duration of the lack of data".

That doesn't work in the general cases where there could be multiple
tracks available, one showing the other speaker, the other showing a
completely different scene.  Switching automatically would be bad.

> The unsolved problem here is to make sure the user agent picks the streams
> in the right order.

Didn't we decide to remove track ordering?  Yes, that is hard.  How
does the browser decide that track A is "better" than track B?  (I
know how we might signal that in SDP, but that isn't the only source
of metainformation about tracks, because RTCPeerConnection isn't the
only source of tracks.

Received on Friday, 6 September 2013 17:10:42 UTC