[whatwg] Cue points in media elements

On 4/29/07, Brian Campbell <Brian.P.Campbell at dartmouth.edu> wrote:
>
> For the sort of content that we produce, cue points are incredibly
> important. Most of our content consists of a video or voiceover
> playing while bullet points appear, animations play, and graphics are
> revealed, all in sync with the video. We have a very simple system
> for doing cue points, that is extremely easy for the content authors
> to write and is robust for paused media, media that is skipped to the
> end, etc. We simply have a blocking call, WAIT, that waits until a
> specific point or the end of a specified media element. For instance,
> in our language, you might see something like this:
>
>    (movie "Foo.mov" :name 'movie)
>    (wait @movie (tc 2 3))
>    (show @bullet-1)
>    (wait @movie)
>    (show @bullet-2)
>
> If the user skips to the end of the media clip, that simply causes
> all WAITs on that  media clip to return instantly. If they skip
> forward in the media clip, without ending it, all WAITs before that
> point will return instantly. If the user pauses the media clip, all
> WAITs on the media clip will block until it is playing again.
>
> This is a nice system, but I can't see how even as simple a system as
> this could be implemented given the current specification of cue
> points. The problem is that the callbacks execute "when the current
> playback position of a media element reaches" the cue point. It seems
> unclear to me what "reaching" a particular time means. If video
> playback freezes for a second, and so misses a cue point, is that
> considered to have been "reached"? Is there any way that you can
> guarantee that a cue point will be executed as long as video has
> passed a particular cue point? With a lot of bookkeeping and the
> "timeupdate" event along with the cue points, you may be able to keep
> track of the current time in the movie well enough to deal with the
> user skipping forward, pausing, and the video stalling and restarting
> due to running out of buffer. This doesn't address, as far as I can
> tell, issues like the thread displaying the video pausing for
> whatever reason and so skipping forward after it resumes, which may
> cause cue points to be lost, and which isn't specified to send a
> "timeupdate" event.
>
> Basically, what is necessary is a way to specify that a cue point
> should always be fired as long as playback has passed a certain time,
> not just if it "reaches" a particular time. This would prevent us
> from having to do a lot of bookkeeping to make sure that cue points
> haven't been missed, and make everything simpler and less fragile.
>
>
In order to capture this kind of situations, with flexibility in mind, I
think the concept of "cue points" may be changed to "cue periods"...

Method names:
addEnterCuePeriod(time1, time2, callback)
removeEnterCuePeriod(time1, time2, callback)
addLeaveCuePeriod(time1, time2, callback)
removeLeaveCuePeriod(time1, time2, callback)

The callback function mentioned by addEnterCuePeriod will be invoked once
when the video enter the period of time bounded by time1 and time2.  How the
video get to a frame between time1 and time2 doesn't matter.  i.e.  the
callback function may be invoked by a normally playing video reaching time1,
a video being fast forward / wind back into the period between time1 &
time2, or a particular timing between time1 & time2 of the video being
directly seek for.

The mechanism of LeaveCuePeriod is similar, while this time the callback is
invoked when the video leave the specified cue period.  (Or should this pair
of methods left out?)

With these four methods, one can not only achieve the "bullet point" effect,
but also video captions appearance and disappearance.

Hope this helps.

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/whatwg-whatwg.org/attachments/20070501/0c68cc27/attachment.htm>

Received on Tuesday, 1 May 2007 08:07:57 UTC