W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > February 2023

Re: [mediacapture-record] Add keyframe interval support. (#216)

From: handellm via GitHub <sysbot+gh@w3.org>
Date: Fri, 03 Feb 2023 14:58:16 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-1415989756-1675436294-sysbot+gh@w3.org>
> Is there a risk with the per-frame API for keyframes, that the encoded stream has more visual artifacts as a result of the encoder having a harder time predicting how to allocate the target bitrate, than if it were to configure the encoder with a per-frame interval?

> Is there specific use case where a time interval is a better fit?

It doesn't have anything to do with per-frame API, but with a fixed bitrate budget, quality per frame becomes lower as the ratio of num keyframe/num deltaframes gets higher. When you think about it this is more likely to happen with time-based control with for example low-framerate slideware. With ultimate control over keyframe generation, this seems like a situation a count-based control makes sense.
(Note, ultimate control isn't entirely possible. Encoders anyway emit keyframes when diffs to reference frames becomes too large, so count-based will not be predictable in this situation.)

> This is the tradeoff I see.

Worth mentioning here's it's also the case that with count-based intervals you can't excise with specified accuracy. Say we've recorded a presentation with occasional slide changes. You'd like to save 0.58 minutes - 1.30 minutes in a webm somewhere else. If the framerate was low in the 0.58 section, with count-based intervals chances are you could have to save an area tens of seconds before that point. 

How about specifying both? Say MR configuration is 2 parameters X (min time between keyframes) and Y (min framecount between keyframes):

void OnFrame(const VideoFrame& frame) {
  now = getCurrentTime();
  forceKeyFrame = now - lastKeyFrameTimestamp >= X &&
      currentFrameIndex - lastKeyFrameIndex >= Y;
  if (forceKeyFrame) {
    lastKeyFrameTimestamp = now;
    lastKeyFrameIndex = currentFrameIndex;
  Encode(frame, forceKeyFrame);

This supports three cases:
1. Pure time-based control (set Y to 0)
2. Pure count-based control (set X to 0)
3. Compromises (X and Y > 0) for variable-rate frame input.

GitHub Notification of comment by handellm
Please view or discuss this issue at https://github.com/w3c/mediacapture-record/pull/216#issuecomment-1415989756 using your GitHub account

Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Friday, 3 February 2023 15:13:38 UTC

This archive was generated by hypermail 2.4.0 : Saturday, 6 May 2023 21:20:00 UTC