- From: guest271314 via GitHub <sysbot+gh@w3.org>
- Date: Fri, 10 May 2019 07:29:49 +0000
- To: public-webrtc-logs@w3.org
@youennf The `MediaRecorder` would necessarily have to replace the track of the underlying media resource (`MediaStreamTrack`) being written (stored) which would lead back to the functionality of `MediaStream`.
Relevant specification language
https://w3c.github.io/mediacapture-main/#dom-mediastream
> 4.1 Introduction
> **A MediaStream is used to group several MediaStreamTrack objects into one unit that can be recorded or rendered in a media element.**
>
> **Each MediaStream can contain zero or more MediaStreamTrack objects. All tracks in a MediaStream are intended to be synchronized when rendered. This is not a hard requirement, since it might not be possible to synchronize tracks from sources that have different clocks.** Different MediaStream objects do not need to be synchronized. (emphasis added)
https://w3c.github.io/mediacapture-fromelement/
> Methods
>
> captureStream
>
> The captureStream() method produces a real-time capture of the media that is rendered to the media element.
>
> The captured MediaStream comprises of MediaStreamTracks that render the content from the set of selected (for VideoTracks, or other exclusively selected track types) or enabled (for AudioTracks, or other track types that support multiple selections) tracks from the media element. If the media element does not have a selected or enabled tracks of a given type, then no MediaStreamTrack of that type is present in the captured stream.
The HTML specification mentions using `videoTrack` property of `HTMLMediaElement`, though as far as am aware (and to the extent have tried) that feature is not possible to implement using Media Fragments URI https://www.w3.org/TR/media-frags/ with `MediaStream` object.
https://www.w3.org/TR/html5/semantics-embedded-content.html#selecting-specific-audio-and-video-tracks-declaratively
> 4.7.13.10.2. Selecting specific audio and video tracks declaratively
> The audioTracks and videoTracks attributes allow scripts to select which track should play, but it is also possible to select specific tracks declaratively, by specifying particular tracks in the fragment of the URL of the media resource. The format of the fragment depends on the MIME type of the media resource. [RFC2046] [URL]
>
> In this example, a video that uses a format that supports the media fragments syntax is embedded in such a way that the alternative angles labeled "Alternative" are enabled instead of the default video track. [MEDIA-FRAGS]
> `<video src="myvideo#track=Alternative"></video> `
`MediaRecorder` specification https://w3c.github.io/mediacapture-record/ does not specifically mention limiting video tracks to 1.
there is ambiguity present amidst the various specifications which use the term `MediaStream` and `MediaStreamTrack`.
Interestingly, Chromium 73 does not dispatch `ended` event at `MediaStreamTrack` when `src` is changed at `HTMLMediaElement`, which technically, should per the specification not stop `MediaRecorder` from recording, though since the audio and video tracks are set at `MediaStream` in order A,A,..V,V or V,V,..A,A, again, it is not possible to use `enabled`, `addTrack` and `removeTrack` or `addtrack` event to continue recording.
Browsing the source code of `MediaRecorder` implementations reveals that multiple tracks have been considered for some time now; e.g., https://github.com/mozilla/gecko-dev/blob/master/dom/media/MediaRecorder.cpp#L778
```
// When MediaRecorder supports multiple tracks, we should set up a single
// MediaInputPort from the input stream, and let main thread check
// track principals async later.
```
https://github.com/mozilla/gecko-dev/blob/master/dom/media/MediaRecorder.cpp#L807
> // We only allow one audio track. See bug 1276928.
> return;
> }
> if (track->AsVideoStreamTrack() && aTrack.AsVideoStreamTrack()) {
> // We only allow one video track. See bug 1276928.
> return;
> }
Similar issues and branches for code which set multiple video tracks that could be selected exist for Chromium; e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=528523.
The relevant issues have already been filed at `MediaRecorder` specification at GitHub (https://github.com/w3c/mediacapture-record/issues/147; et al., briefly referenced at the first post).
Would not characterize the lack of support for recording multiple `MediaStreamTrack`s of kind video as a browser implementation "limitation"; the expected result is possible using various workarounds; where the functionality, from perspective here, should be possible directly at the `MediaStream` itself, to avoid the need to to address the potential for continued ambiguity at specifications derived from or using the `MediaStream` language. Allowing pause and resume between `src` changes to an `HTMLMediaElement` where the `timecode` EBML elements generated would be in sequential order would also be another possible solution. Though `replaceTrack()` appears to achieve the expected result, save for the issue of the last one second or less of recorded audio being muted.
Do not have a particular preference for how the functionality of using `MediaRecorder` to record multiple tracks is implemented - the more different ways the functionality is implemented, the more ways there will bbe possible to actually create code when outputs the same result at FOSS browsers. Since `replaceTrack` is already implemented, and the WebRTC developers are active, the incorporation of `replaceTrack` into the `MediaStream` and capture API should not be particularly difficult; less difficult in actual code than in writing exacting technical documents which actually are consistent in definitions and actual implementations.
--
GitHub Notification of comment by guest271314
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/586#issuecomment-491188615 using your GitHub account
Received on Friday, 10 May 2019 07:29:51 UTC