Re: Proposal for Audio and Video Track Selection and Synchronisation for Media Elements

On Thu, 24 Mar 2011, Philip Jägenstedt wrote:
> On Wed, 23 Mar 2011 03:10:51 +0100, Ian Hickson <ian@hixie.ch> wrote:
> > On Tue, 22 Mar 2011, Philip Jägenstedt wrote:
> > > > 
> > > > Well if you know ahead of time which track you want you can just 
> > > > use fragment identifiers in the 'src' attribute, but yes, worse 
> > > > case it would look something like this:
> > > > 
> > > >  var controller = new MediaController();
> > > >  var video1 = document.createElement('video');
> > > >  video1.controller = controller;
> > > >  video1.src = 'video.foo';
> > > >  video1.onloadedmetadata = function () {
> > > >    video1.videoTracks.select(1);
> > > >  };
> > > >  var video2 = document.createElement('video');
> > > >  video2.controller = controller;
> > > >  video2.src = 'video.foo';
> > > >  video2.onloadedmetadata = function () {
> > > >    video2.muted = true;
> > > >    video2.videoTracks.select(2);
> > > >  };
> > > > 
> > > > 
> > > > > This seems like it would work, but what if steps 2 and 3 happen 
> > > > > in the other order?
> > > > 
> > > > So long as they happen in the same task, it doesn't matter; the 
> > > > loading of the media resource happens asynchronously after the 
> > > > task ends.
> > > 
> > > It's inevitable that someone will set video2.controller in 
> > > video2.onloadedmetadata.
> > 
> > Sure, but by that point they have much bigger problems (e.g. the 
> > videos will be playing out of sync).
> 
> I assumed that when you assign a common controller to two videos, they 
> would be synchronized from then on, isn't that the case?

They would have been sychronised starting with the relative offset that 
they had when they were synchronised, not necessarily synchonised to the 
same playback position, because the MediaController just ensured a common 
clock, not a common playback position, and their playback position could 
have been different at the time the controller was set.

But I have changed the API to handle this case; now there is no way to 
offset multiple tracks from each other. We can add an explicit offset 
control later should this be found to be useful.


> > > In your proposal, the decoding of two video streams from the same 
> > > resource is not as tightly coupled, one can only make educated 
> > > guesses.
> > 
> > There's no guesswork. You know it's the same resource, and you know 
> > they're being synchronised. What is there to guess about?
> 
> The problem is that for a single resource there are basically two modes 
> of operation:
> 
> 1. n streams playing in sync at the same playback rate and offset.
> 
> 2. n streams playing at different playback rates or offsets, although 
> possibly still synchronized by a controller.

There's no way to do #2 now in this proposal. Hopefully that addresses 
your concern.


> Synchronized looping just seems like a somewhat odd feature that adds
> implementation complexity, so I'd like to see some concrete use cases.

I've made the spec block looping, since it has implementation complexity 
and no compelling use cases.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'