Re: Tech Discussions on the Multitrack Media (issue-152)

On Mon, 21 Feb 2011 15:13:02 +0100, Silvia Pfeiffer  
<> wrote:

[snip, snip, sniperoo]

>>> Another reason for having a default display is when you go
>>> full-screen. It would be hard if not impossible to create the layouts
>>> for fullscreen display for multitrack video without the browser taking
>>> care of it.
>> I think something like <>
>> handles this quite nicely by allowing any element to go fullscreen, so  
>> just
>> make the common parent of all your <video> elements fullscreen.
> That is possible, but seems counter-intuitive, when already there are
> right-click interfaces to video that allow you to go fullscreen.

A fullscreen mode that only sets the <video> element to fullscreen  
wouldn't even allow for custom controls. In the end, web authors will have  
to put some minimal effort into making interfaces (fullscreen or not) that  
make sense for multitrack video.

>> The samples you collected and random searches on Google images seems to
>> indicate that more often than not, something more fancy than 1px solid
>> black/white is used.
> If I had collected user interfaces for controls, I would have ended up
> with fancy ones, too. I wasn't actually looking very hard and took the
> first ones that showed the principle. I don't believe in the necessity
> for excessive styling here, but was only looking out for a common
> approach to display multitrack video. We definitely need to find more
> examples though.

A fair point. Still, since it is seemingly common, we should support fancy  
layout of multitrack video, just like we support custom controls. The  
question is how much added complexity we can afford to get default  

>>> I have been thinking lots about side-by-side video and how to display
>>> them.
>>> If they are separate boxes on the Web page, that makes it difficult to
>>> have a single @controls to navigate across them. I think the
>>> complexity of multiple @controls may be too hard to deal with. Thus,
>>> right now I tend towards the idea that they should all be displayed in
>>> a single viewport. If a side-by-side display is requested and we have
>>> two video tracks, then they would sit next to each other with
>>> letterboxing on top and bottom. That would also provide space for
>>> captions to be moved into.
>>> When we have side-by-side and 3 video tracks, I would think there's
>>> two next to each other, one centered below them and pillarboxing
>>> around it.
>>> When we have side-by-side and 4 tracks, they would get tiled, etc.
>>> If the sizes don't match, appropriate letter- and pillarboxing would be
>>> used.
>>> Scaling would need to be within the viewport given for the main video
>>> - either through CSS or through the width/height of the main video
>>> track. This is the base viewport size and within that the tiling would
>>> happen by subdividing the viewport size into equal size rows and
>>> columns and adequately letter- / pillarboxing. That should also allow
>>> to reuse most of the scaling code that is already in use for the
>>> single-track videos.
>> That would be possible, but my main question is "is it worth it"? Also,  
>> what
>> would happen to .videoWidth and .videoHeight?
> The biggest problem I have with separate video elements displaying the
> multitrack (virtual) video resource is that of the controls and that
> they are not obviously linked to each other, visually, other than if
> the user decides to make it that way. So, we may end up with multiple
> video elements each with @controls, but only one of them is able to
> introduce and manage the @controls activities. I find such a situation
> very disturbing.

Examples of existing solutions would be helpful here, too. One kind of  
approach that comes to mind is to allow redirecting the rendering of the  
default controls to another element, one that encompasses all the video  
elements. This is similar to the idea of directing the rendering of text  
tracks to some other element than the <video> element itself. It's a bit  
iffy and certainly not without issues, but perhaps worth considering.

> If instead we can have the connection between the elements, but a
> display that is within one video viewport, then I would favor such a
> solution.
> Instead, when they are all inside the video viewport, that makes it
> nice and easy. The .videoWidth and .videoHeight would continue to
> display the intrinsic width and height of the element, so in the case
> of 4 video tracks displayed in parallel, it might end up to be a
> quarter of the video viewport size. Das that create further problems?

videoWidth and videoHeight are defined to reflect the intrinsic width and  
height, which by definition is the size of the video content box unless it  
is restricted by CSS. We would have to choose between letting  
videoWidth/Height reflect the size of the video stream (which one?) or of  
the actual size of content box (redundent with getComputedStyle). I think  
both options are rather weird, things seem cleaner if one <video> element  
represents the state of just one video stream.

Philip Jägenstedt
Core Developer
Opera Software

Received on Monday, 21 February 2011 15:10:05 UTC