Re: [mediacapture-record] Add replaceStream to MediaRecorder (#186)

> Discussed this with @Pehrsons and we have some concerns with this API.
> 
> While browsers today appear limited to recording only one video track and one audio track, there's no such limitation in the spec, so we think we need to consider how to handle several tracks.
> 
> Say someone records a stream like this:
> 
> ```js
> const stream = new MediaStream([videoTrack, englishAudioTrack, frenchAudioTrack]);
> const recorder = new MediaRecorder(stream);
> recorder.start();
> ```
> 
> How would they replace just the french audio? Tracks in a MediaStream unfortunately are unordered.
> 
> So we'd like to propose this instead:
> 
> ```js
> await recorder.replaceTrack(frenchAudioTrack, directorsCommentaryTrack);
> ```

I'll just comment on this that one can trivially polyfill the functionality of `replaceStream` with `replaceTrack` (modulo replacing the recorder's `stream` attribute). But polyfilling `replaceTrack` with `replaceStream` becomes hacky as one would have to replace one track at a time.

> In addition, we propose to no longer stop the recorder if a stream's track-set changes, because it no longer makes sense to react to those. Instead, we'd copy the track-set in `start()`, and only stop on the first track `ended`.
> 
> If the tracks' isolation properties differ, `replaceTrack` would reject with `SecurityError`.
> 
> If`stop()` followed by `start()` is called again then the stream can be examined anew.

I thought some more about the need to for the `replace` method to return a promise. The only argument I've heard in favor of returning a promise is that then you know you can dispose of the old tracks without risking ending them in the recording before the new ones take effect.

I think this is unnecessary. The MediaRecorder could guarantee that `recorder.replaceTrack(oldTrack, newTrack); oldTrack.stop();` replaces the track successfully as long as the track is not ended.

To stretch things a bit, the recorder could guarantee that the track is replaced successfully as long as the recorder is not in the `inactive` state.

This would help applications handle concatenating/swapping tracks when the application itself is not in charge of the lifetime of those tracks, like HTMLMediaElement's `captureStream()`.

In fact, if one of two tracks the recorder is recording has ended, there's nothing to say in the spec that the recording of the ended track (let's call it the output track) has also ended. The spec says
> If **_all_** recorded tracks become ended, then stop gathering data, and queue a task (...) Inactivate the recorder

(emphasis mine)

Though this highlights a race that would need to be fixed, in the case of:
```
const rec = new MediaRecorder(stream);
rec.start();
await new Promise(r => rec.onstart);
stream.getTracks().forEach(t => t.stop()); // stops gathering data and queues a task to inactivate
rec.replaceTrack(stream.getTracks()[0], newTrack); // rec.state is not inactive yet -- should this throw? -- or abort the task queued above?
```
Perhaps that'd be the reason we need to return a promise.

-- 
GitHub Notification of comment by Pehrsons
Please view or discuss this issue at https://github.com/w3c/mediacapture-record/pull/186#issuecomment-531702341 using your GitHub account

Received on Monday, 16 September 2019 09:25:41 UTC