[media-source] Interop issues when an append overlaps currentTime (#373)

ntrrgc has just created a new issue for https://github.com/w3c/media-source:

== Interop issues when an append overlaps currentTime ==
Let's suppose an application calls `appendBuffer()` with a presentation range that includes the `currentTime`. The most typical scenario is that the browser was playing quality A and quality B was just appended. What should happen?

* Approach A: In WebKit, decoding of quality A is stopped (i.e. a flush happens) and decoding of quality B starts immediately.
    * Advantages: The content shown by the HTMLMediaElement video element is **always synchronized** with the latest state of the frames fed to MSE.
    * Disadvantages: Playback is **briefly interrupted**, resulting in a potentially noticeable hiccup. How noticeable this is depends on how fast the decoder and the rest of the browser playback pipeline can catch up.
* Approach B: In Chromium, of quality A continues, followed by decoding of quality B.
    * Advantages: The quality change can be **smooth**, at least in typical scenarios.
    * Disadvantages: The transition between quality A and B happens potentially later than expected. It is unclear how long-GOP content could be handled under this approach, if at all.
* (There are other potential hypothetical approaches, like decoding both qualities simultaneously and switching when the newer catches up, although they can pose significant implementation challenges, particularly with some embedded hardware decoders.)

Currently, as far as I know, which approach is used is a user-agent implementation detail.

Unfortunately, players in the wild do appends in a way that overwrite the currentTime and time near to it.

This is at the very least the case in a **default setup of dash.js** with Fast Switching ABR enabled (the default), and it causes hiccups in browsers using approach A that are noticeable in heavily constrained devices.

### Why this is tricky for applications

During MSE playback there are three relevant time values:
* `video.currentTime`, represents the presentation time that is being displayed to the user.
* `video.buffered.end(0)` (or some other index) represents the highest presentation time the browser has been fed by the application (in all  necessary SourceBuffers).
* `highestEnqueuedPresentationTime` (using WebKit terminology): the highest presentation time the browser has enqueued for playback in some implementation-specific manner at that time that cannot be reverted without some sort of flush operation. This can range from a few frames to a few seconds.
  Conversely, any append where all the presentation times are greater than this can be done without any flush or delay, even if there is an overlap with the buffered ranges.

A player that wants to avoid the hiccups would need to make sure that all presentation timestamps in the new append are greater than `highestEnqueuedPresentationTime`, but it cannot know that value, as it's an implementation detail (also, it can change during the execution of the script). Best they can do is take an educated guess of it being the `currentTime` plus some fixed number of seconds.

There is also an interesting race presented by quality changes to such an application implementation:
1. The player realizes bandwidth has increased and so it decides to switch to a new quality.
2. Trying to do the switch relatively quickly but still avoid stalls, it requests the segment corresponding to currentTime + 5 seconds.
3. Due to server or network issues, the response is unexpectedly delayed by a couple seconds. By the time the JavaScript application receives it, it is too close to currentTime to append it without risk of hiccups. Should the application risk the stall, or delay the `appendBuffer()` until the new data is well behind currentTime?

### Questions

* Should which approach is used by the user-agent remain an implementation detail, or should it be limited by the spec in some way?
* If it remains an implementation detail, should the spec provide some non-normative guidance for applications?

Please view or discuss this issue at https://github.com/w3c/media-source/issues/373 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 12 May 2026 20:31:06 UTC