[media-source] Coded Frame Removal algorithm steps need fixing to match actual intent (#290)

wolenetz has just created a new issue for https://github.com/w3c/media-source:

== Coded Frame Removal algorithm steps need fixing to match actual intent ==
There are several related problems with the text for this algorithm as it exists/existed in the v1 REC spec:

* Current steps 3.3.2-5 should actually be substeps of 3.3.1, not steps on the same level as 3.3.1. The intent of those substeps is to inform the coded frame processing algorithm of a discontinuity caused by removal of a frame that was just (successfully) processed. This discontinuity shouldn't be signalled on *every* execution of the coded frame removal algorithm, but instead only when the condition in step 3.3.1 occurs.

* Current step 3.3.1 ("For each removed frame, if...") does not consider that there may be multiple buffered coded frames with different PTS but same DTS in a track buffer, since those DTS may have originated from different coded frame groups. (DTS is really an ordering mechanism within a group of frames having decode dependency on each other back to the last random access point.) Different splices in the buffered timeline for a track may, for instance, each start their independent coded frame groups with the same DTS, but they are distinct, and a removal of just one of those groups shouldn't always trigger discontinuity handling in the coded frame processing algorithm -- that should only happen if the most recently (successfully) processed frame is removed by this algorithm's step 3.3.

* Current step 3.4 should also be able to conditionally trigger discontinuity handling: if the last (successfully) processed frame was not removed in 3.3, but is dependent on frame(s) removed in 3.3, then last decode timestamp needs unsetting on all track buffers, last frame duration needs unsetting on all track buffers, highest end timestamp needs unsetting on all track buffers, and need random access point flag needs setting to true on all track buffers.

Note: Chromium has for a long time implemented behavior as if these steps were corrected, as above. I suspect other implementations also do, so this may be borderline editorial, but certainly needs discussion and fixing.

Please view or discuss this issue at https://github.com/w3c/media-source/issues/290 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 4 August 2021 20:27:24 UTC