Re: Tech Discussions on the Multitrack Media (issue-152)

On Feb 18, 2011, at 2:08 AM, Philip Jägenstedt wrote:

> On Thu, 17 Feb 2011 18:43:49 +0100, Mark Watson <watsonm@netflix.com>  
> wrote:
> 
>> 
>> On Feb 17, 2011, at 7:17 AM, Philip Jägenstedt wrote:
>> 
>>> On Wed, 16 Feb 2011 18:47:22 +0100, Mark Watson <watsonm@netflix.com>
>>> wrote:
>>> 
>>>> 
>>>> On Feb 16, 2011, at 12:02 AM, Philip Jägenstedt wrote:
>>>> 
>>>>> On Wed, 16 Feb 2011 03:31:47 +0100, Silvia Pfeiffer
>>>>> <silviapfeiffer1@gmail.com> wrote:
>>>>> 
>>>>>> On Wed, Feb 16, 2011 at 12:08 PM, Jonas Sicking <jonas@sicking.cc>
>>>>>> wrote:
>>>>>>> On Tue, Feb 15, 2011 at 4:19 PM, Silvia Pfeiffer
>>>>>>> <silviapfeiffer1@gmail.com> wrote:
>>>>>>>> On Wed, Feb 16, 2011 at 5:36 AM, Mark Watson <watsonm@netflix.com>
>>>>>>>> wrote:
>>>>>>>>> Hi Philip,
>>>>>>>>> 
>>>>>>>>> Just a quick note that the "alternative" vs "additional"  
>>>>>>>>> distinction
>>>>>>>>> is not always completely clear. Video with different camera angles
>>>>>>>>> (gimmiky or not) could be considered as an alternative, or could  
>>>>>>>>> be
>>>>>>>>> rendered as picture-in-picture, or multiple thumbnail videos could
>>>>>>>>> show beside the main video (some sports sites already do this kind
>>>>>>>>> of
>>>>>>>>> thing).
>>>>> 
>>>>> Sure, but all of those modes should be achieved by the author making  
>>>>> it
>>>>> happen with CSS. At the risk of making a strawman argument, I honestly
>>>>> can't see browsers allowing the user to change the rendering of the  
>>>>> page
>>>>> to achieve PiP or something like that when the author hasn't provided
>>>>> for
>>>>> it, messing with the layout like that seems both weird and unlikely to
>>>>> be
>>>>> useful. Of course we can have User JavaScript and User CSS to do that
>>>>> kind
>>>>> of thing, though.
>>>> 
>>>> I was assuming that the "author" of the content - who labels the tracks
>>>> - might not be the same as the "author" of the webpage that is  
>>>> rendering
>>>> the content. So the first author should not assume that (say) multiple
>>>> views are alternatives, because some webpages might be able to view  
>>>> them
>>>> both as PIP.
>>> 
>>> Since the tracks are labeled using the attribute of the <track>  
>>> attribute,
>>> it will be the page author that has to do the work to support some
>>> specific video display, be that PiP, overlay or something else.
>> 
>> That would be the case for track objects created as a result of <track>  
>> elements, but what about in-band tracks ? The page author does the work  
>> for PIP etc., of course, but the media author should not assume that  
>> such capabilities are or are not available on the pages where their  
>> media might be used: they should just label the tracks and let the page  
>> to whatever it is capable of.
> 
> I don't think we should spend much time making extra in-band video tracks  
> work more than barely, if at all, since the extra bandwidth needed to have  
> multiple in-band video tracks makes it quite unlikely the feature would be  
> used to any greater extent.

A track declared within an adaptive streaming manifest (e.g. a DASH manifest or take-your-pick of various proprietary adaptive streaming solutions) would be an in-band track but would only be fetched when actually needed. 

> 
> If they should work at all, my position is that the only thing you should  
> be able to do with in-band video tracks is switch between them, in other  
> words what I've called alternative tracks. Either having some kind of  
> layout information in the file itself or having HTML markup to target  
> individual tracks of the same resource seems like unjustified complexity  
> and spec/implementation effort not very well spent.

I think people do imagine that adaptive streaming manifests would declare all the tracks needed for a presentation - including sign language tracks that are additional rather than alternative. Such manifests have to be useful in environments other than HTML and so need to included everything. I don't think we should ask people to re-author them in HTML for use in HTML environments.

I guess what I am saying is that Option (1) in the wiki write-up should be supported in order to provide support for adaptive streaming. The questions are:
(1) whether this should be the only way to declare such additional/alternative tracks or whether an HTML markup way is also required (and I think that it is)
(2) what should that markup be
(3) how to define the API for discovering and manipulating these tracks in a way that is common for in-band (from a file or from an adaptive streaming manifest) and explicitly marked up tracks.

...Mark

> 
> -- 
> Philip Jägenstedt
> Core Developer
> Opera Software
> 
>

Received on Friday, 18 February 2011 16:09:27 UTC