Re: Rationalizing new/start/end/mute/unmute/enabled/disabled

On 4/3/13 5:29 PM, Randell Jesup wrote:
> On 3/25/2013 5:55 PM, Martin Thomson wrote:
>> I think that it's fair to say that the current state of the
>> MediaStream[Track] states is pretty dire.  At least from a usability
>> perspective.
>
> I generally agree with the approach here.  I also agree that the
> MediaStream should be an explicit rollup of the states of the tracks
> (just as I feel we need a stream.stop() in addition to track.stop(),
> even though you can build one in JS yourself).
>
> One thing I really want to see described (doesn't have to be in the
> spec) is how an application can provide a "Hold" operation where live
> video is replaced with a video slate and/or pre-recorded
> animation/audio, and do it without an offer/answer exchange.  The
> MediaStream Processing API would have made this fairly simple, but since
> we don't have that, we need to define something.  WebAudio (if/when
> implemented) may help (with some pain) for the audio side, but doesn't
> help us with video.
>
> The real blocker here is giving a way for a MediaStreamTrack (in a
> MediaStream that's already AddStream()ed to a PeerConnection) to get a
> different source.  Currently, the only easy way I can see it is very
> kludgy and probably higher overhead/delay than we'd like:
>
>         video_hold.src = my_hold_animation.ogg;
>         elevator_music = video_hold.captureStreamUntilEnded();

What is this? An undocumented feature of the media element?

>         source = getUserMedia();
>         video.srcObject = source;
>         stream = video.captureStreamUntilEnded();
>         pc.addStream(stream);
>         video.play();
>         ...
>         <user hits Hold>
>         video.srcObject = elevator_music;
>         video_hold.play();
>
> and the same to switch back.  Putting up a video slate on video mute
> (but keeping audio alive) is more fun...
>
> This sequence makes me sad...  And will induce extra delay in all
> likelihood, since you need to route through two mediastreams and a media
> element for normal operation.
>
> The only alternative I see in the current spec might be to have two
> tracks always, and disable the live track and enable the Hold/slate
> track - but that would still cause a negotiationneeded I believe before
> it took affect.

I will not argue that being able to switch source for a MediaStreamTrack 
is useless, because I think it could be useful. But switching source 
could very well also lead to that a renegotiation is needed (I take this 
from what Cullen said in Boston: if the current source encodes with 
codec A but the other with codec B you'd have to renegotiate anyway).

If I understand the use-case you describe here correctly, you'd like to 
switch to Muzak being played out at the remote end when you mute locally.

The straghtforward way of meeting this use-case with the current set of 
APIs would be:

1. As the application loads, download the Muzak source file

2. Use one video element to render the live a/v stream from the remote 
party, and a separate (muted) audio element that has Muzak as source - 
set up in a "loop" fashion

3. When the user at the remote end presses "mute me" in the application, 
have the app a) disable the audio track and b)send a "play Muzak" signal 
to the peer

4. When "change to Muzak" is received, unmute the Muzak audio element 
(no need to mute the video element as silence is being played)

5. Same goes for unmute -> signal "stop play Muzak" -> mute the Muzak 
audio element.

There are many other options as well.



>
> p.s. For Hold and Mute in a video context, I rarely if ever want "black"
> as an application.  I may want to send information about Hold/Mute
> states to the receiver so the receiver can do something smart with the
> UI, but at a minimum I want to be able to provide a
> slate/audio/music/Muzak-version-of-Devo's-Whip-It (yes, I heard that in
> a Jack In The Box...)

There is also the "poster" possibility with the video element (i.e. an 
image to be played in absence of video).

>

Received on Thursday, 4 April 2013 11:02:15 UTC