[Bug 27242] Clarify how track buffer ranges are updated.

https://www.w3.org/Bugs/Public/show_bug.cgi?id=27242

--- Comment #5 from Aaron Colwell <acolwell@google.com> ---
(In reply to Matt Wolenetz from comment #3)
> I'm noting some further things I believe need clarification in track buffer
> range calculation in the spec:
> 
> By example:
> 
> 1) Append a video keyframe buffer A, whose presentation interval is roughly:
>    [A..............)
> 2) Append another video keyframe buffer B, whose presentation interval is
> contained completely within A's: (B's PTS) >= ((A's PTS) + (1 microsecond))
> and (B's PTS+duration) < (A's PTS+duration):
>       [B......)
> 
> What should be the track's buffered ranges at this point? Should A's
> duration be truncated to join the end of A with the beginning of B?
> Ambiguity exists: the spec isn't clear whether the track buffer contains:
>    [A.[B......)....) ---> Render A, then B, then A again? Unlikely this is
> desired.
> Or:
>    [A)[B......) ---> Render A, then B, then done (and possible gap is
> introduced until next buffered range).

This second option is what I'd expect to happen since this is essentially an
overlap. This second option seems consistant with the audio behavior to me.

> 
> A further complexity is introduced if the initial append were followed by
> some dependent (non-key) frames:
>    [A..............)[a1.....][a2.....][a3.....]
> After appending B, the impact of the spec ambiguity increases. Should the
> track buffer then contain:
>    [A.[B......)....)[a1.....][a2.....][a3.....] ---> Render keyframe A, then
> keyframe B, then back to keyframe A again, then dependent frames a1, a2, and
> a3? Unlikely this is desired, especially if both A and B ended at the same
> time.
> Or:
>    [A)[B......) ---> Render A, then B, then done (and even more likely there
> is a gap introduced until next buffered range).

The second option seems appropriate here as well. The way I see it, you are
essentially talking about an overlap situation here. Since a frame is being
inserted between A and a1, the decode dependency chain is essentially broken so
ISTM that this is essentially equivalent to other overlap scenarios.

> 
> What should the buffered result be if an app issues Remove() to remove
> exactly the presentation interval for B? I think the sanest approach might
> be just:
>    [A) --> just the first tiny bit of A, then a potentially even larger gap.
> 

Seem reasonable to me. 

I think these 2 issues should be moved into a separate bug though. This bug was
originally intended to just specify how the contents of the track buffer be
represented as a TimeRanges so the algorithms that reference the data buffered
in track buffers was well defined. What you are talking about here has more to
do with modifying the "coded frame processing" algorithm to truncate coded
frames and remove stuff from the track buffer.

> Finally, the spec is not clear regarding how much of a "gap" can be
> introduced (in audio or video tracks) before a previously contiguous
> buffered range is split in two. In general, how close to a range must a
> coded frame group's presentation interval be to be considered continuous,
> contained within, that range? The coded frame processing algorithm describes
> this clearly for parsing a stream of new frames, but the spec is unclear
> regarding exactly how other operations like scattered appends, overlapped
> appends, removes, etc result in buffered range(s).

In general, I believe the "coded frame group" concept captures this since they
represent a group of adjacent frames that aren't considered to have any gaps.
Step 6 of the "coded frame processing algorithm" somewhat addresses this as it
determines what triggers the beginning of a new "coded frame group". I agree
that there is likely unspeced behavior that Chrome has to essentially merge
adjacent "coded frame groups" that are "close enough" to each other. In general
I think this was intended to be an extension of the "2 frame duration" rule
that appears in step 6. I definitely think address that issue should be a
separate bug.

> At least one implementation (Chromium) tracks a maximum inter-frame distance
> for each track and uses this in a heuristic to determine range
> membership/continuity.
Yeah. This will probably need to be added to the spec in some form just to
insure interoperability when appends occur in random order. I would encourage
you to take a fresh look at this and try not to be biased by Chrome's current
implementation. We definitely need some form of heuristic here, but the
presentation global max inter-frame distance may not be the best option
especially when mixing content with different frame rates. Perhaps a max within
the current coded frame group might be a better alternative. I don't know.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Friday, 3 April 2015 16:11:56 UTC