[mediacapture-record] Ability to record non-realtime / frame-by-frame (#213)

akre54 has just created a new issue for https://github.com/w3c/mediacapture-record:

== Ability to record non-realtime / frame-by-frame ==
Hi all - I'm trying to capture a canvas with expensive draw computations (e.g. variable-length network I/O and complex 3D rendering) asynchronously, separating out the render work from the capture. Essentially I'm looking for a way to separate out the video container's timestamp from the wall timestamp, similar to the CCapture library.

Reduced example:

```js
async function loop(t) {
  const props = interpolator(t);
  await render(props);
  await capture();
}
```

The [Media Capture from DOM Elements spec says](https://www.w3.org/TR/mediacapture-fromelement/#dom-htmlcanvaselement-capturestream) that a call to `canvas.captureStream` with a frameRate of 0 allows users to add frames to the stream manually with `track.requestFrame()` and [this comment](https://github.com/w3c/mediacapture-record/issues/148#issuecomment-405568103) seems to suggest that the MediaRecorder simply reads output from the stream. But in practice it appears as if the MediaRecorder spec records in real-time, using the wall clock and leading to a choppy output video.

I tried to call the `pause` and then `resume` MediaRecorder methods in my `capture` method before and after the `requestFrame`, but this seemed to create a corrupt output.

Example: https://jsfiddle.net/akre54/71aonkeb/ - Change the delay value for `setTimeout` on line 65.

What currently happens: the captured video reflects the content of the canvas as it was drawn in real-time (i.e. choppy and with the setTimeout delays incorporated). What should happen: the output video is always the same duration, with smooth playback.

Any suggestions for how to accomplish this? Thanks!

Related: #177 #114

Please view or discuss this issue at https://github.com/w3c/mediacapture-record/issues/213 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Sunday, 12 June 2022 12:52:33 UTC