W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > June 2022

[mediacapture-record] Ability to record non-realtime / frame-by-frame (#213)

From: Adam Krebs via GitHub <sysbot+gh@w3.org>
Date: Sun, 12 Jun 2022 12:52:32 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issues.opened-1268572387-1655038350-sysbot+gh@w3.org>
akre54 has just created a new issue for https://github.com/w3c/mediacapture-record:

== Ability to record non-realtime / frame-by-frame ==
Hi all - I'm trying to capture a canvas with expensive draw computations (e.g. variable-length network I/O and complex 3D rendering) asynchronously, separating out the render work from the capture. Essentially I'm looking for a way to separate out the video container's timestamp from the wall timestamp, similar to the CCapture library.

Reduced example:

async function loop(t) {
  const props = interpolator(t);
  await render(props);
  await capture();

The [Media Capture from DOM Elements spec says](https://www.w3.org/TR/mediacapture-fromelement/#dom-htmlcanvaselement-capturestream) that a call to `canvas.captureStream` with a frameRate of 0 allows users to add frames to the stream manually with `track.requestFrame()` and [this comment](https://github.com/w3c/mediacapture-record/issues/148#issuecomment-405568103) seems to suggest that the MediaRecorder simply reads output from the stream. But in practice it appears as if the MediaRecorder spec records in real-time, using the wall clock and leading to a choppy output video.

I tried to call the `pause` and then `resume` MediaRecorder methods in my `capture` method before and after the `requestFrame`, but this seemed to create a corrupt output.

Example: https://jsfiddle.net/akre54/71aonkeb/ - Change the delay value for `setTimeout` on line 65.

What currently happens: the captured video reflects the content of the canvas as it was drawn in real-time (i.e. choppy and with the setTimeout delays incorporated). What should happen: the output video is always the same duration, with smooth playback.

Any suggestions for how to accomplish this? Thanks!

Related: #177 #114

Please view or discuss this issue at https://github.com/w3c/mediacapture-record/issues/213 using your GitHub account

Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Sunday, 12 June 2022 12:52:33 UTC

This archive was generated by hypermail 2.4.0 : Saturday, 6 May 2023 21:19:57 UTC