W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > September 2019

Re: [mediacapture-record] Add replaceTrack method to MediaStream (#167)

From: guest271314 via GitHub <sysbot+gh@w3.org>
Date: Fri, 13 Sep 2019 00:57:37 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-531060600-1568336255-sysbot+gh@w3.org>
@henbos Since Media Capture and Stream does not define any order for `MediaStreamTrack`s within a `MediaStream` the order is arbitrary both between instances of `MediaRecorder` at the same browser and between Firefox and Chromium, Chrome. There must be some form of algorithm to replace the correct track in the existing media stream. This is the second version of the algorithm for merging WebM tracks output by `MediaRecorder` at Chromium after having created the corresponding A or V track if not encoded in the original stream, e.g, [A0] (input) = [A0,V0] (create V0 for input A0) + [V1,A1] (append V1 to V0, A1 to V0 note arbitrary track order) + [V2] (input) => [A2, V2] (create audio track for V2) => [A_0, V_0] // output

  ```
  // https://www.reddit.com/r/mkvtoolnix/comments/cdi824/mkvmerge_prints_error_when_merging_webm_files/etwhugs/
    // 1. run `mkvmerge -J` file.webm on each input file
    // 2. determine which tracks go together
    // 3. create an appropriate --append-to argument from the algorithm in 2
    let filesMetadata = [];
    for (const fileName of fileNames) {
      await new Promise(resolve => {
        const getMetadata = ({
          body
        }) => {
          port.onMessage.removeListener(getMetadata);
          filesMetadata.push(JSON.parse(body));
          resolve();
        };
        port.onMessage.addListener(getMetadata);
        port.postMessage({
          "message": "metadata",
          "body": `${cmd} ${metadata} ${fileName}`
        });
      });
    };
    const tracks = filesMetadata.map(({tracks}) => tracks);
    const trackList = tracks.map(track => ({audio:getTrack(track, "audio").id, video:getTrack(track, "video").id}));
    // check if tracks are ordered AV,AV...AV or arbitrarily AV,VA,AV,AV,VA...AV
    const orderedTracks = tracks.every(([{type}]) => type === "audio");
    // construct `--append-to` option for merging files where
    // tracks are not in consistent order; for example, WebM
    // files output by Chromium, Firefox MediaRecorder implementations 
    // Chromium => Opus: "id": 0, Firefox => Opus: "id": 1,
    // Chromium => VP8: "id": 1, Firefox => VP8: "id": 0 
    // merging a WebM file output by MediaRecorder at Chromium 
    // with a WebM file produced by MediaRecorder at Firefox crashes tab
    trackList.reduce((a, b, index) => (
      appendTo += `${index}:${b.audio}:${index-1}:${a.audio},${index}:${b.video}:${index-1}:${a.video}${trackList[index+1]?",":""}`
      , b
    ));
    console.log(trackList, JSON.stringify({
      filesMetadata, orderedTracks, appendTo
    }, null, 2));
    port.onMessage.addListener(onNativeMessage);
    const message = {
      "message": "write",
      // if tracks in files are not ordered use `--append-to` option, else do not
      "body": `${cmd} ${options} ${outputFileName} ${!orderedTracks ? appendTo : ""} '[' ${fileNames.join(" ")} ']'`
    };
    // mkvmerge <35 outputs error when merging video tracks having differing pixel dimensions
    // Error: The track number 1 from the file '1.webm' cannot be appended to the track number 
    // 1 from the file '0.webm'. The width of the two tracks is different: 768 and 480
    // fixed with mkvmerge: allow appending video tracks with differing pixel dimensions 
    // https://gitlab.com/mbunkus/mkvtoolnix/commit/2548a7f3497940135e55be604a87b9495fdff94d
    port.postMessage(message);
```
output https://next.plnkr.co/plunk/8J61Rw. (see https://github.com/pthatcherg/web-codecs/issues/11#issuecomment-529552845)

-- 
GitHub Notification of comment by guest271314
Please view or discuss this issue at https://github.com/w3c/mediacapture-record/issues/167#issuecomment-531060600 using your GitHub account
Received on Friday, 13 September 2019 00:57:38 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:22:28 UTC