Re: Tech Discussions on the Multitrack Media (issue-152)

On Sat, 19 Feb 2011 23:35:13 +0100, John Foliot <jfoliot@stanford.edu>  
wrote:

> Mark Watson wrote:
>>
>> On Feb 18, 2011, at 2:08 AM, Philip Jägenstedt wrote:
>> >
>> > I don't think we should spend much time making extra in-band video
>> tracks
>> > work more than barely, if at all, since the extra bandwidth needed to
>> have
>> > multiple in-band video tracks makes it quite unlikely the feature
>> would be
>> > used to any greater extent.
>>
>> A track declared within an adaptive streaming manifest (e.g. a DASH
>> manifest or take-your-pick of various proprietary adaptive streaming
>> solutions) would be an in-band track but would only be fetched when
>> actually needed.
>
> This has been an interesting conversation.
>
> Philip, I think we need to be careful about the assumption you made, as
> from an accessibility best-practices perspective, ensuring all supporting
> media (be it textual or binary) is best included as in-band content, for
> the very same reason why providing textual (captioning) data in-band is
> preferable: portability and re-use. Isn't this why we worked on getting
> the JavaScript API ready early on?
>
> While I concede that the inclusion of sign language interpretation and
> descriptive audio may seem edge-case compared to the larger body of
> content envisioned to be on the web, it is important that we ensure we  
> can
> do this, and do it both well and properly. Thus I think we need to spend
> as much time as required to ensure we *have* met this requirement, and I
> am a tad concerned that we suggest that content such as this *not* be
> treated in the same way as textual supporting content.

Certainly, accessibility improvements in the form of extra audio and video  
tracks is the main use case here, so failing to achieve that is not an  
option. In those environments where extra video tracks for accessibility  
have already been rolled out, is it really the case that all video tracks  
are muxed in a single file (and sent over a slowish network)? Concrete  
examples of the current state of the art would be much appreciated. It  
seems to me that delivering all video tracks in a single file would be  
waste bandwidth to the point that people just won't do it, except for  
situations where a large majority of users are expected to use all of the  
tracks.

My gut feeling is that most of the time people will want to use separate  
video tracks to save bandwidth and have some control over how the video  
tracks are laid out on the page. For this, using multiple <video> elements  
is a perfect fit. Since we'll most certainly need a solution in that  
general direction, it would be great if we could get away with *only* that  
single solution. If that turns out to be impossible then so be it, we'll  
see where we land after another 1000 mails or so.

> The idea that a DASH manifest would only fetch this type of content
> 'on-demand' is intriguing; however does it not presume an active
> connection to the network? Or would the DASH manifest also be used to
> 'activate' or expose supporting in-band content such as sign language
> content, etc. to the user-agent?

I assume this part was not directed at me, but the answer would interest  
me.

-- 
Philip Jägenstedt
Core Developer
Opera Software

Received on Sunday, 20 February 2011 16:48:43 UTC