Re: Feedback on MediaController

On Apr 20, 2011, at 4:01 PM, Ian Hickson wrote:

> On Mon, 18 Apr 2011, Philip Jägenstedt wrote:
>> 
>> I think that videoTracks should be an ExclusiveTrackList, to avoid having to
>> invent a layout system for displaying multiple video streams in a single
>> <video>. Even if WebM supported positioning video tracks similar to what
>> MPEG-4 presumably can, this layout will likely not be good enough in most
>> cases anyway, as you can't do fancy borders and likely couldn't do things like
>> move the videos around by mouse dragging in JavaScript.
> 
> Indeed.
> 
> It's also unclear to me what it would mean to enable multiple in-band 
> video tracks and then disable them in the opposite order, if the last one 
> enabled had a different geometry than the first one. What would the 
> rendering be? I don't mind the UA exposing this to the user, but from the 
> perspective of the script it seems like a rather complicated problem.
> 
  For a QuickTime movie, videoWidth and videoHeight are calculated from the union of the bounding boxes of all enabled visual tracks.  Therefore, videoWidth and videoHeight will change if disabling a track changes the union rect. This is true in Safari today. 


>> (3) looping should be possible on combined multitrack:
>> 
>> In proposal 4 the loop attribute on individual media elements is
>> disabled on multitrack created through a controller, because it is not
>> clear what looping means for the individual element.
>> 
>> However, looping on a multitrack resource with in-band tracks is well
>> defined and goes over the complete resource.
> 
> It's not especially well-defined, since there's no concept of "ending" 
> with the controller, given how streaming is handled.
> 
> But more importantly, what are the use cases?
> 
> The use case for looping a single track is things like this:
> 
>   http://www.google.com/green/
> 
> ...but I don't see why you would use a MediaController to do that kind of 
> thing. It's not like you'd want the multiple videos there in sync, they're 
> just background.
> 
  We have had several requests from developers who want to create something very much like http://www.google.com/green/, but with the background videos playing in sync (and looping). The developers essentially want a non-rectangular video, which in theory could be done with a single file with alpha, but alpha is currently not widely supported in video formats.


> I'm also skeptical of introducing loop at the MediaController level even 
> in the simple case of finite resources, because it's not clear how to make 
> it work with looping subresources. Say you had two resources, both set to 
> loop, one of which was 5s and one 3s, and that you then further say that 
> the whole thing should loop. What should happen? We don't want to define 
> MediaController looping in a way that precludes that from being possible, 
> IMHO, at least not unless we have a strong use case.
> 
  MediaController.currentTime would be set to 0 when it reaches the duration of the longest subresource.


>> * onended: is raised when all  slave media elements are in ended state
> 
> This isn't supported; a media controller can't be "ended" currently.
> 
  It could be "ended" when currentTime reaches the duration of the longest subresource.


> On Mon, 18 Apr 2011, Silvia Pfeiffer wrote:
>> 
>> Onended is important to do something once the video or audio resource is 
>> finished playing, such as display related videos, or display a post-roll 
>> ad.
> 
> That seems like something you'd want to do on a per-<video> basis, not for 
> the whole MediaController, surely.
> 
  It depends on why the MediaController is being used. In the case of the example you used earlier of a director's commentary on a movie, you would want whatever happens at the "end" to occur when the *presentation* ends, not one of its parts. 

  Without an 'ended' event from the MediaController, a script will have to listen for the event on all elements and calculate when the presentation "ends". Not impossible, but It will be much simpler if the MediaController fires this event.



> On Wed, 20 Apr 2011, Silvia Pfeiffer wrote:
>>> 
>>> Looking at the metadata list cited above, I don't see anything in 
>>> either ogg, mp4, or webm that maps to "kind", so I don't see much 
>>> point exposing that on the audio/video track lists, though I agree 
>>> that in principle it would be a good idea.

  For MPEG-4 and QuickTime files, the mapping can sometimes be made from a track's media format. For example, a closed caption track has 'clcp' sample data, 'tx3g' is MPEG-4/3GPP Timed Text, etc. It is also possible for a track's metadata to list its "kind".


>>>> We discussed the looping behaviour. To make it symmetrical with 
>>>> in-band multitrack resources, it would make sense to be able to loop 
>>>> over composed multitrack resources, too. The expected looping 
>>>> behaviour is that a loop on the composed resource loops over the 
>>>> composite as a whole. So, the question is then how to turn such 
>>>> looping on.
>>>> 
>>>> The proposal is that when one media element in the group has a @loop 
>>>> attribute, that would turn the looping on the composite resource on. 
>>>> This means that when the loop is set and the end of the composite 
>>>> resource is reached (its duration), the currentTime would be reset to 
>>>> its beginning and playback of the composite resource would start 
>>>> again. Looping on individual elements is turned off and only the 
>>>> composite resource can loop.
>>> 
>>> What's the use case?
>> 
>> The same as for the loop attributes on a audio or video element.
> 
> I don't see how MediaController applies to those use cases. For example, 
> for audio, a use case is background music in a game. No controller. For 
> video, a use case is the effect seen on the aforementioned Google page. 
> Again, no controller.
> 
  See my previous comment about looping multiple synced background videos.


>> It's not easily possible to stop hand-coded play() calls, which would be 
>> the only way for grouped multitrack media in the way in which it is 
>> currently specified.
> 
> As currently specified, grouped tracks always autoplay any tracks that are 
> labeled autoplay, and those that are not simply don't play (while the 
> controller's position advances regardless).
> 
  I think it will be confusing for developers that some tracks in a group can play while others do not. It seems more logical to me that the playback state of all elements in a group should be kept in sync.

eric

Received on Thursday, 21 April 2011 16:44:45 UTC