[mediacapture-record] Ability to tell MediaRecorder to maximize frame rate (#177)

j0000el has just created a new issue for https://github.com/w3c/mediacapture-record:

== Ability to tell MediaRecorder to maximize frame rate ==
Videos produced via the MediaRecorder API have highly inconsistent frame rates. Using the same stream settings and MediaRecorder configuration, captured videos can range from average frame rates that are good (30ish, assuming stream frame rate is 30) to bad (below 20). While inconsistent frame rates could be acceptable in the real-time communication case, they are not acceptable if you're building a camera app that captures video with the Stream/MediaRecorder APIs. 

The issue is most pronounced when using a mobile device (tests done in my case with Pixel 1 and 3 on Chrome). MediaRecorder seems to prefer delivering frames in real-time, so if encoding a frame takes too long, it simply drops it and moves on to the next. Since mobile devices are slower, potentially more likely to be overloaded, I believe the issue is much more pronounced there.

Rather than delivering frames as fast as possible, dropping frames if it can't keep up, I would like to see the ability to tell MediaRecorder to prefer a higher, more consistent frame rate on the encoded video blob, even if the final blob lags a bit after MediaRecorder.stop() is called (dataavailable won't be fired immediately, for instance, until encoding is complete).

I can see this as a new option that can be passed in the constructor. Passing a target frame rate is likely not a great option, as that would be dependent on the input stream frame rate and the browser should likely retain some flexibility to drop frames if the device is severely overloaded (even native camera apps drop frames, just not as bad as MediaRecorder on the same device). Perhaps just a hint to the encoder that more time should be dedicated to each frame before skipping to the next, potentially causing a lag in delivering video segments to the client. My naming-foo is not strong here, so open to suggestions, but something like a boolean property named 'videoMaximizeFrameRate' (default false).

const mediaRecorder = new MediaRecorder(stream, {
  mimeType: 'video/webm',
  videoMaximizeFrameRate: true,
// Some time passes.
mediaRecorder.ondataavailable = () => {
  // Video blob delivered after some delay, with encoded frame rate as close to input
  // stream frame rate as possible.

Please view or discuss this issue at https://github.com/w3c/mediacapture-record/issues/177 using your GitHub account

Received on Thursday, 1 August 2019 21:43:56 UTC