Re: [mediacapture-record] Input width, height MUST be recorded and playable (#172)

> It seems reasonable to expect the video to be recorded with dynamically changing resolution, so that input frames are correctly represented in the output. The main issue I see though, is if recording into a container that doesn't support changing resolution dynamically. Should that be an error? Or should it be left to the UA to take an alternative approach, like scale, crop, etc?
> 
> I don't know the container landscape well enough to say whether, or which, containers support this, but I imagine there are some that don't. Others might know better.

AFAIK the only container that Firefox and Chromium support is Matroska (multiple codecs supported), or Webm (Matroska with VP8, VP9, was Vorbis, now Opus). The codecs that those browsers' implement contains code which checks the width and height of input.

This is the edge case (observable at Chromium) which led to filing this issue https://bugs.chromium.org/p/chromium/issues/detail?id=972470 and this PR. Perhaps there is a bug in the code. Can you get the recording the output the width and height changes at Chromium?

```
<!DOCTYPE html>
<html>
<head>
</head>
<body>
  <video id="sink" autoplay muted controls></video>
  <script>
    (async() => {
      const sink = document.querySelector("video");
      const canvas = document.createElement("canvas");
      const imageData = [{
        width: 50,
        height: 50,
        fillStyle: "green"
      }, {
        width: 150,
        height: 150,
        fillStyle: "yellow"
      }, {
        width: 300,
        height: 300,
        fillStyle: "red"
      }];
      const segments = 60;
      const videoData = [];
      const ctx = canvas.getContext("2d");
      const canvasStream = canvas.captureStream(0);
      const [videoTrack] = canvasStream.getVideoTracks();
      const mediaStream = [canvasStream, videoTrack].find(({requestFrame:rF}) => rF);
      const recorder = new MediaRecorder(canvasStream);
      const result = new Promise(resolve =>
        recorder.ondataavailable = e => {
          sink.load();
          sink.srcObject = null;
          resolve(sink.src = URL.createObjectURL(e.data));
        });
      recorder.start();
      videoTrack.onunmute = e => console.log(e);
      videoTrack.onmute = e => console.log(e);
      videoTrack.onended = e => console.log(e);
      sink.onloadedmetadata = e => console.log(e.type, e.target.tagName, sink.videoWidth, sink.videoHeight);
      sink.onresize = e => console.log(e.type, e.target.tagName, sink.videoWidth, sink.videoHeight);
      sink.srcObject = canvasStream;

      for (const {width, height, fillStyle} of imageData) {
        // At Chromium 76 the only code that have been able to set width and height with
        // results in the incorrect value for height: 150 when width: 300
        // for last 60 frames, should be width: 300 when height: 300 
        // ({width:canvas, height:canvas});
        // Object.assign(canvas, {width, height});
        console.log(canvas.width, canvas.height, {width, height});
        // at Firefox 69 each set width and heiht of canvas
        // is reflected at resulting webm video by settin width and height
        // this should be the only setting needed and output the same result at Chromium
        canvas.width = width;
        canvas.height = height;
        await new Promise(async resolve => {
          for (let i = 0; i < segments; i++) {
            await new Promise(promise =>
              requestAnimationFrame(function draw() {
                ctx.clearRect(0, 0, width, height);
                ctx.fillStyle = fillStyle;
                ctx.fillRect(0, 0, width, height);
                mediaStream.requestFrame();
                promise();
              })
            )
          }
          resolve()
        })
      }
      recorder.stop();
      videoTrack.stop();
      videoTrack.enabled = false;
      console.log(await result, sink.srcObject, canvasStream, videoTrack);
    })();
  </script>
</body>
</html>
```
Interestingly, while experiementing with code other than the edge at Chromium the width and height is recorded when `<canvas>` `width` and `height` is changed.

If a container does not support variable width the container should not be implemented by `MediaRecorder`. 

Had not considered whether or not an error should be thrown. At first impression, variable width and height input should "just work", else the entire concept of `canvas.captureStream()` needs to be revisted and constraints should be deleted from the several revelant specifications which state they could remotely have an effect on a `MediaStreamTrack`. The error would be with the implementation by browser authors. Though if your question is essentially if an error should not be thrown then what is the point of this PR then yes, throw the error; which would require some internal code and testing to verify that the output width and height corresponds to the input width and height - which should already be implemented. 

-- 
GitHub Notification of comment by guest271314
Please view or discuss this issue at https://github.com/w3c/mediacapture-record/pull/172#issuecomment-506397242 using your GitHub account

Received on Thursday, 27 June 2019 15:35:25 UTC