RE: CHANGE: Use a JS Object as an argument to getUserMedia

Hi

Some comments inline and some general thougts at the end.

Timothy B. Terriberry wrote:

> Adam Bergkvist wrote:
>> empty (no tracks), and tracks would have to be added later. I think
>> it would simplify things (e.g. MediaStream playback and sending with
>> PeerConnection) if a MediaStream is immutable with regards to its
>> track list.
> 
> I'm not sure this is really a problem: the request indicates
> whether you asked for audio and/or video, and tracks can be
> pre-created that simply don't reach the appropriate ready
> state until the user gives consent (if ever). You can still
> argue about whether you want the user to be able to consent
> to "just audio" or "just video" when you asked for both, and
> what should be done in that case. I'll let Anant tackle that issue.

A stream may contain any number of audio and video tracks which makes it hard to pre-create tracks even if you know what types the stream may contain. The question about "just audio" and "just video" is tricky. On one hand, it's not good if a user can't use an application because it requests video but the user doesn't have a camera, but on the other hand, the application may behave strangely if an expected component is missing.

> 3) Remove the old MediaStream and add a new MediaStream
> containing both the old MediaStreamTrack corresponding to the
> audio and the new MediaStreamTrack corresponding to the video.
> 
> When you do this, you can either
> a) literally use the same MediaStreamTrack object used in the
> old MediaStream, or
> b) create a new MediaStreamTrack object for the old audio
> track in the new MediaStream.
> 
> I think 3a would mean, for example, that if you ignored the
> callback and continued to use the old MediaStream object,
> then the audio would continue playing through it. That leaves
> you with an object that _acts_ is if it was one of the
> currently active remote streams, but is not actually in the
> PeerConnection's list of remote streams. It also means it may
> not be synchronized with the new track, unless you do
> something to enforce that synchronization (e.g., switch to
> using the new MediaStream object).

If the A-side removes a stream it will be "finished" (in the ENDED state) on the B-side and cannot be used anymore, according to the spec. This would be the case for both 3a and 3b.

> So, that's as far as I've thought through these things right
> now. What do others think?

It's possible to do option 2 and 3 with the current API, and any of those could be the option of choice for a web developer even if we support adding new tracks to a stream. Perhaps two ways to achieve this is enough and they need to work fairly well anyhow? Other MediaStream consumers like MediaStreamRecorder, the media elements and media processing APIs would also need to handle new tracks being added on the fly.

An other way to support adding video to an audio conversation would be to initially ask for audio and video and keep the video track disabled until the user requests to add video to the conversation. The "add video" option would not be available if only audio was selected. This reduces the number of queries to the user, but may give the page access to more than it will use (if video never gets used).

/Adam

Received on Thursday, 6 October 2011 16:39:30 UTC