Re: Rationalizing new/start/end/mute/unmute/enabled/disabled

On 3/25/2013 5:55 PM, Martin Thomson wrote:
> I think that it's fair to say that the current state of the
> MediaStream[Track] states is pretty dire.  At least from a usability
> perspective.

I generally agree with the approach here.  I also agree that the 
MediaStream should be an explicit rollup of the states of the tracks 
(just as I feel we need a stream.stop() in addition to track.stop(), 
even though you can build one in JS yourself).

One thing I really want to see described (doesn't have to be in the 
spec) is how an application can provide a "Hold" operation where live 
video is replaced with a video slate and/or pre-recorded 
animation/audio, and do it without an offer/answer exchange.  The 
MediaStream Processing API would have made this fairly simple, but since 
we don't have that, we need to define something.  WebAudio (if/when 
implemented) may help (with some pain) for the audio side, but doesn't 
help us with video.

The real blocker here is giving a way for a MediaStreamTrack (in a 
MediaStream that's already AddStream()ed to a PeerConnection) to get a 
different source.  Currently, the only easy way I can see it is very 
kludgy and probably higher overhead/delay than we'd like:

        video_hold.src = my_hold_animation.ogg;
        elevator_music = video_hold.captureStreamUntilEnded();
        source = getUserMedia();
        video.srcObject = source;
        stream = video.captureStreamUntilEnded();
        pc.addStream(stream);
        video.play();
        ...
        <user hits Hold>
        video.srcObject = elevator_music;
        video_hold.play();

and the same to switch back.  Putting up a video slate on video mute 
(but keeping audio alive) is more fun...

This sequence makes me sad...  And will induce extra delay in all 
likelihood, since you need to route through two mediastreams and a media 
element for normal operation.

The only alternative I see in the current spec might be to have two 
tracks always, and disable the live track and enable the Hold/slate 
track - but that would still cause a negotiationneeded I believe before 
it took affect.

p.s. For Hold and Mute in a video context, I rarely if ever want "black" 
as an application.  I may want to send information about Hold/Mute 
states to the receiver so the receiver can do something smart with the 
UI, but at a minimum I want to be able to provide a 
slate/audio/music/Muzak-version-of-Devo's-Whip-It (yes, I heard that in 
a Jack In The Box...)

-- 
Randell Jesup
randell-ietf@jesup.org

Received on Wednesday, 3 April 2013 15:31:32 UTC