MediaStreams and state changes

For discussion as part of the constraints issues today:

I have some unease about some parts of MediaStreams and getUserMedia 
that we haven't addressed or haven't discussed in detail.

One is changing video parameters after getUserMedia().

Another is changing parameters of a stream that originates somewhere 
other than getUserMedia.  video elements using 
captureStreamUntilEnded(), PeerConnection-sourced streams (which might 
want to renegotiate based on the request, or send RTCP feedback, or use  
application-specific signaling), algorithmic generators (such as the 
Audio API), etc.

Right now, we've  somewhat waved our hands and said there'd be a method 
to re-do constraints after a MediaStream is created.  I generally am 
uneasy with this, and I'm especially uneasy when we're not talking about 
a getUserMedia()-originated stream.

The code that originated the stream might be separated from the code 
consuming it that wants a change, so we need some way for a consumer of 
a MediaStream to indicate a need or request for changes in it.

Concerning re-doing the constraints call via a method on the mediastream 
(which had been vaguely proposed when we discussed this last):
   a) re-doing the constraints has problems: the consumer of the stream 
may not be the caller to getUserMedia.  It might not even know who was.
   b) implementing the full constraint algorithm and structure on all 
sources of MediaStreams is a considerable burden and might not match well

If instead we pass a request back "upstream" (nominally as some form of 
event), it would let the event ripple up to the source (or sources) - 
this would apply at the track level generally.  There may be multiple 
MediaStream objects and processors between the consumer and the source.

Also, fundamental object for this type of change is the track, and not 
the MediaStream, which needs to be thought about even if we were to use 
constraints for modifications.

My apologies for not having a concrete proposal; we don't have one on 
the table now either, and the vague comments make me worry we may leave 
this off too long (and the solution to it may impact the start-time 
constraints discussion).

-- 
Randell Jesup
randell-ietf@jesup.org

Received on Thursday, 7 June 2012 14:18:34 UTC