Re: [MSE] Bug 18615 - HTMLMediaElement.buffered behavior in "ended" state

As mentioned on the call, if we think of endOfStream() as placing a marker in each source buffer, indicating the time at which each stream ends, then I think we get consistent answers to the questions below.

With a global endOfStream() call, the marker is placed at the end of the frame with the latest timestamp in each source buffer.

Calling endOfStream() would not mean that no more data will be appended, it means that no more data with a later timestamp than the latest existing timestamp will be appended.

On Jan 10, 2013, at 10:02 AM, Aaron Colwell wrote:

Hi,

I'd like to restart the discussion around Bug 18615<https://www.w3.org/Bugs/Public/show_bug.cgi?id=18615> so we can get closer to closing it out. The gist of the problem is how should the buffered property and playback behavior change when the MediaSource enters the ended state.

For well behaved applications that append multiplexed data the behavior is relatively straight forward and I think the existing spec text works fine. The problems appear when audio & video are appended with different SourceBuffers and buffered regions of each SourceBuffer may be disjoint in several regions when endOfStream() is called.

Here are some questions to start the discussion:

1. What should the behavior be if there are gaps between the current playback position and the end of the buffered data when endOfStream() is called? For example, the current position is 10 and SourceBuffer.buffered reports [0, 15) and [20, 30).

1a. Should playback continue through this gap?

No, playback should stall at the gap.

1b. Should the duration be truncated to the range that contains the current position?

No. The state endOfStream() should be considered an indication of where the media of the stream ends, not an indication that nothing more will be appended - the missing data might get append (we allow out-of-order appends).

1c. Should an exception be thrown by endOfStream() if the current playback position isn't in the last range?

No.

1d. Should a "network error" be signalled when the current position reaches the end of the range? This is roughly equivalent to an HTMLMediaElement being unable to fetch data it needs.

No, just a normal playback stall, waiting for that missing data.


2. If multiple SourceBuffers are being used and they don't contain roughly the same buffered ranges when endOfStream() is called, then how should playback proceed? Philip give an example in the bug where we have a video SourceBuffer with the following ranges [0, 10], [20,30]  and an audio SourceBuffer with the range [0,15]

Playback of only one of the two media types can continue passed the endOfStream() point of one of them.

So, playback could continue with only video for [15,30] if endOfStream() was called in the state above, provided the video media for [15,30] is (becomes) available.


2a. Should playback continue to the highest buffered data

Yes.

and play through the gaps?

No.

2b. Should the buffered data be truncated to the intersection of the ranges?

No.


This is just the questions I can think of off the top of my head. These situations don't really happen in traditional HTMLMediaElement playback so I think we are blazing new trails here.

The corner case which is not addressed by a global endOfStream() (compared to per source buffer endOfStream()) is the following: suppose the audio stream ends at 15s and the video ends at 20s. Suppose I have audio for [0,15] and video for [0,18]. I can't yet call a global endOfStream() because I have not appended the latest bit of video data. Suppose now playback gets to 15s. Playback will stall, whereas it should continue through [15,18] with only the video.

If I had been able to call a audio endOfStream() the player would know there its no audio data after 15s and could continue playing through to 18s, but which time the remaining 2s of video might have arrived.

…Mark



Aaron