Re: MediaStreams and state changes

On 08/02/2012 10:07 AM, Randell Jesup wrote:
> On 8/1/2012 3:54 PM, Travis Leithead wrote:
>>> -----Original Message----- From: Randell Jesup
>>> [mailto:randell-ietf@jesup.org] Sent: Thursday, June 7, 2012 7:18
>>> AM
>>>
>>> For discussion as part of the constraints issues today:
>>>
>>> I have some unease about some parts of MediaStreams and
>>> getUserMedia that we haven't addressed or haven't discussed in
>>> detail.
>>>
>>> One is changing video parameters after getUserMedia().
>> This is relevant to the other thread I started
>> (http://lists.w3.org/Archives/Public/public-media-capture/2012Jul/0069.html)--sorry,
>> somehow I missed this particular thread. I did, however, want to
>> share some thought here as well as they seem relevant to the
>> general conversation.
>
> I did also raise this in one of the Telco's, but I'm not sure you
> were on that one.
>
>>> Another is changing parameters of a stream that originates
>>> somewhere other than getUserMedia.  video elements using
>>> captureStreamUntilEnded(), PeerConnection-sourced streams (which
>>> might want to renegotiate based on the request, or send RTCP
>>> feedback, or use application-specific signaling), algorithmic
>>> generators (such as the Audio API), etc.
>>>
>>> Right now, we've  somewhat waved our hands and said there'd be a
>>> method to re-do constraints after a MediaStream is created.  I
>>> generally am uneasy with this, and I'm especially uneasy when
>>> we're not talking about a getUserMedia()-originated stream.
>> So, now we have a proposal on the table for this. I took care to
>> isolate the proposed API to a LocalMediaStream, and essentially
>> punted on the option to allow downstream consumers to be able to
>> apply change requests (directly through an API). I'd originally
>> wanted to tie the API to the MediaStreamTrack, but then you allow
>> any consumer (downstream or upstream) to be able to request
>> changes, and these probably aren't relevant for Tracks that come
>> from PeerConnection.
>
> These sorts of requests are frequently applicable to
> PeerConnections. The simplest version of such a request, a request to
> change resolution, would make total sense for PeerConnection-sourced
> streams - in fact we're trying to standardize the wire protocol for
> doing this (currently it appears there's consensus for doing it in
> SDP).  Others might not be handled by the system directly, but an
> application could catch the requests and broker them to the other
> party over DataChannels (or signaling). For example, think of a
> WebRTC-enabled security system where you could switch cameras, tell
> cameras to change parameters, etc.

If we isolate this down to resolution of video, I had this idea of that 
the consumer would hint to the producer the size, and the producer could 
adapt. Say you have a MediaStream connected to a video element, the 
width/height of the video element could be used to change the resolution 
produced by the camera - the browser knows has all facts about the 
consumer and controls the producer. Similarly a record function would 
have to provide width/size. If there are several consumers (e.g. two 
local video elements, or one video element along with a recording) the 
browser could determine the best compromise (which probably is to catch 
video with the highest resolution consumed).

This idea could easily be extended over a PeerConnection. If the video 
is consumed on the other side of the PeerConnection the width/height 
could be signaled back to the sending browser, and used as input to how 
that browser handles the camera.

The advantage would be that the naive author would just have to set up 
the size of the video element to fit the intended layout, and the rest 
would be automatic.

However, one feedback has been that this should not 'just happen', the 
app should be in control. So perhaps it is not a good idea.

But if the application controls the resolution, is there really a 
benefit if the app receiving a MediaStream from a PeerConnection can ask 
(using some local API) for a specific resolution? Could this not just as 
well be signaled back to the sending app, that uses (a variant of) the 
API proposed by Travis to change the resolution produced by the camera.

Received on Sunday, 5 August 2012 18:03:27 UTC