Re: Dropping cues when playbackRate != 1.0

On Sat, Aug 17, 2013 at 3:17 AM, Eric Carlson <eric.carlson@apple.com>wrote:

>
> On Aug 8, 2013, at 7:21 PM, Silvia Pfeiffer <silviapfeiffer1@gmail.com>
> wrote:
>
> > On Fri, Aug 9, 2013 at 6:11 AM, Brendan Long <self@brendanlong.com>
> wrote:
> >> There's section of the HTML5 CR spec saying:
> >>
> >> Similarly, when the playback rate is not exactly 1.0, hardware,
> software, or
> >> format limitations can cause video frames to be dropped and audio to be
> >> choppy or muted.
> >>
> >>
> >> I think this should also apply to text track cues, something like:
> >>
> >> Similarly, when the playback rate is not exactly 1.0, hardware,
> software, or
> >> format limitations can cause video frames and text cues to be dropped
> and
> >> audio to be choppy or muted.
> >>
> >> When playing at non-standard speeds, an efficient implementation may
> want to
> >> skip portions of the file, which could mean skipping cues.
> >
> > I assume you are talking about a situation where the playback would
> > need to jump multiple seconds (call them "s") at a time to achieve the
> > speedup.
> > And you are further assuming that when the current time jumps from
> > second x to second x+s, there may be a cue that would have both their
> > start and end time between x and x+s.
> > So you're saying that if there is no audio or video rendered to which
> > the cue refers, the "time marches on" algorithm doesn't actually
> > activate these cues and therefore they are "dropped".
> >
> > In actual fact, the "time marches on" algorithm will activate the
> > enter and exit events of the "missed cues", so they are not really
> > "dropped". They are, however, not rendered, because they don't become
> > active.
> >
>   This requires that all of the media data for time X to X+S has to be
> loaded before skipping ahead.


Why? Only the media data for time x and for time x+s has to be loaded -
nothing else.



> This means that a UA can not play at a faster rate by not loading the
> entire file, for example by only loading and displaying key frames.


>   If this is allowed, and it seems silly to prevent it, cues defined in
> the portions of the file that are not loaded would have to be skipped.
>

I guess we have two different kind of cues: those coming from <track> and
those coming from inside a media file.

Those from a <track> element are all loaded and events could be activated
as the timeline skips over the cue start/end times. Cue content would be
rendered when X or X+S is within a cue's interval.

Those from inside a media file, if multiplexed, may get completely skipped
by going from X to X+S without the browser loading those parts of a file.
So, in this case you'd probably skip cues.


I wonder if that may be a problem for in-band metadata tracks where, for
example, the passing of cues are supposed to trigger a video pause and
display of some other content (such as a video ad).


Silvia.