Re: timing model of the media resource in HTML5

On Nov 25, 2009, at 2:49 PM, Philip Jägenstedt wrote:

> On Wed, 25 Nov 2009 22:49:40 +0100, Eric Carlson <eric.carlson@apple.com 
> > wrote:
>
>>
>> On Nov 25, 2009, at 12:02 PM, Philip Jägenstedt wrote:
>>
>>> On Wed, 25 Nov 2009 18:43:39 +0100, Eric Carlson <eric.carlson@apple.com 
>>> > wrote:
>>>
>>>>
>>>>  I think <overlay> should be used for internal subtitle and/or  
>>>> closed caption tracks as well. Further, I think that we will want  
>>>> them to "just work" so a UA should create an <overlay> element if  
>>>> the markup doesn't have one and it finds that a file has internal  
>>>> captions/subtitles:
>>>>
>>>>    <video src='my-captioned-movie'> </video>
>>>>
>>>
>>> Yes, that sounds good. One issue is how to style such an implicit  
>>> <overlay>. Should one actually include an <overlay> in the markup  
>>> and somehow indicate that it can/should be used to render in-band  
>>> subtitles from the resource?
>>>
>>> <video src="my-captioned-movie">
>>> <caption style="font-weight:bold" magic-attribute></caption>
>>> </video>
>>>
>>> Not awesome. Perhaps a new CSS pseudo-selector could be used?  
>>> Other ideas?
>>>
>>   Actually I was imagining that *all* subtitles and captions, in- 
>> band and external alike, would be rendered into an <overlay>. If  
>> the markup doesn't includes an <overlay> element the UA would  
>> actually insert one into the DOM, as is done now for other missing  
>> elements (eg. tbody, etc). This way the default style could be  
>> specified in the user agent style sheet and the author could  
>> override as they wish.
>>
>
> Right, all subtitles/captions should be rendered in <overlay>  
> regardless of origin. If the parser automatically inserts an  
> <overlay> element when there is none, what about the case where  
> there is an <overlay> used to show custom controls? I imagined the  
> need for a magic attribute/CSS selector/something to point out the  
> correct <overlay> in cases like this. Possibly a magic src  
> attribute? In any case, these are small issues that I'm sure could  
> be sorted out if <overlay> is implemented.
>
>>   Speaking of author overrides, another issue we need to deal with  
>> is authors that wish to handle captions themselves. Posting an  
>> event when a new caption needs to be displayed seems logical  
>> enough, but how do we provide access to the caption data in  
>> JavaScript?
>>
>
> Here I think we should do something similar to cue ranges as has  
> been discussed before in various places. A new event type would  
> allow us to add some data, e.g.
>
> interface CueRangeEvent : Event {
>  readonly attribute double startTime;
>  readonly attribute double endTime;
>  readonly attribute DOMString text;
> };
>
> We would need to bring back addCueRange with some modifications.
>
> v.addCueRange(10 /* start */, 12 /* end */, "Hello");
> v.addEventListener('cuerangeenter', function(e)  
> { e.target.querySelector('overlay').textContent = e.text; }, false);
> v.addEventListener('cuerangeleave', function(e)  
> { e.target.querySelector('overlay').textContent = ''; }, false);
>
> You get the idea even if the above doesn't have the perfect  
> interface/method/event names. Something along these lines should  
> make it possible to handle in-band, external and script-created  
> captions in a quite uniform fashion, as well as provide for whatever  
> use cases the old cue range API had.

This interface works ok for the specific case of popping up some text,  
but it seems like it would be awkward for anything more complicated,  
since there is only a single event and set of handlers. What I would  
suggest is the declarative cue range idea that was suggested on the  
whatwg list a while back:

<video>
     <source type="video/mp4" src="video.m4v">
     <timerange start="10" end="12" onrangeenter="enterRange1()"  
onrangeleave="leaveRange1()">
</video>

This makes it really easy to have different handlers per cue range  
without having to express that difference as a string. It also makes  
it simpler to use cue ranges for two orthogonal purposes.

addCureRange() could just be a shortcut for adding such a <range>  
element:

var range = v.addCueRange(10, 12);
range.addEventListener("rangeenter", function(e)  
{ e.target.querySelector('overlay').textContent = "Hello"; }, false);
range.addEventListener("rangeleave", function(e)  
{ e.target.querySelector('overlay').textContent = ''; }, false);

Another possibility is that <timerange> elements have contents which  
automatically become visible or hidden depending on whether content is  
in the range, so the common use case (make some content appear during  
certain time ranges of the video) work without any script:

<video>
     <source type="video/mp4" src="video.m4v">
     <timerange start="10" end="12">Hello</timerange>
</video>

The contents could be arbitrary HTML, which would make it very simple  
to sync a slideshow to a video, in addition to handling the captions  
use case. CSS styling could be used to position the currently visible  
<timerange> over the video.

Regards,
Maciej

Received on Saturday, 28 November 2009 18:42:51 UTC