Re: [mediacapture-record] Add keyframe interval support. (#216)

> Is there a risk with the per-frame API for keyframes, that the encoded stream has more visual artifacts as a result of the encoder having a harder time predicting how to allocate the target bitrate, than if it were to configure the encoder with a per-frame interval?

> Is there specific use case where a time interval is a better fit?

It doesn't have anything to do with per-frame API, but with a fixed bitrate budget, quality per frame becomes lower as the ratio of num keyframe/num deltaframes gets higher. When you think about it this is more likely to happen with time-based control with for example low-framerate slideware. With ultimate control over keyframe generation, this seems like a situation a count-based control makes sense.
(Note, ultimate control isn't entirely possible. Encoders anyway emit keyframes when diffs to reference frames becomes too large, so count-based will not be predictable in this situation.)

> This is the tradeoff I see.

Worth mentioning here's it's also the case that with count-based intervals you can't excise with specified accuracy. Say we've recorded a presentation with occasional slide changes. You'd like to save 0.58 minutes - 1.30 minutes in a webm somewhere else. If the framerate was low in the 0.58 section, with count-based intervals chances are you could have to save an area tens of seconds before that point. 

How about specifying both? Say MR configuration is 2 parameters X (min time between keyframes) and Y (min framecount between keyframes):

```
void OnFrame(const VideoFrame& frame) {
  ++currentFrameIndex;
  now = getCurrentTime();
  forceKeyFrame = now - lastKeyFrameTimestamp >= X &&
      currentFrameIndex - lastKeyFrameIndex >= Y;
  if (forceKeyFrame) {
    lastKeyFrameTimestamp = now;
    lastKeyFrameIndex = currentFrameIndex;
  }
  Encode(frame, forceKeyFrame);
}
```

This supports three cases:
1. Pure time-based control (set Y to 0)
2. Pure count-based control (set X to 0)
3. Compromises (X and Y > 0) for variable-rate frame input.


-- 
GitHub Notification of comment by handellm
Please view or discuss this issue at https://github.com/w3c/mediacapture-record/pull/216#issuecomment-1415989756 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 3 February 2023 15:13:38 UTC