Re: [mediaqueries4] Media queries for multichannel audio ?

On Mar 20, 2013, at 11:38 AM, Sylvain Galineau wrote:

> 
> 
> On 3/20/13 11:14 AM, "Mark Watson" <watsonm@netflix.com> wrote:
> 
>> 
>> On Mar 20, 2013, at 10:22 AM, Tab Atkins Jr. wrote:
>> 
>>> On Wed, Mar 20, 2013 at 10:19 AM, Mark Watson <watsonm@netflix.com>
>>> wrote:
>>>> On Mar 20, 2013, at 10:10 AM, Tab Atkins Jr. wrote:
>>>>> On Wed, Mar 20, 2013 at 8:37 AM, Mark Watson <watsonm@netflix.com>
>>>>> wrote:
>>>>>> What I need is just the device width and height in device pixels, to
>>>>>> know
>>>>>> when it is not worth delivering a high resolution version of the
>>>>>> content.
>>>>>> It's not such a big deal with HD, since there are a dwindling number
>>>>>> of SD
>>>>>> displays, but with 4K this will be an issue.
>>>>> 
>>>>> You don't want that.  What you want instead is some way to declare the
>>>>> resolution on the <source> element, so (1) the browser can
>>>>> intelligently choose which source to download, based on what it knows
>>>>> about the display, the user's internet connection, and possibly other
>>>>> things, and (2)  the browser can offer both options to the user, a la
>>>>> YouTube's resolution selection.
>>>> 
>>>> I'm interested in implementing the resource selection algorithm in
>>>> Javascript for the case where I am using the Media Source Extensions,
>>>> not
>>>> the <source> element. See
>>>> https://www.w3.org/Bugs/Public/show_bug.cgi?id=21336.
>>> 
>>> Forgive me for not finding that a compelling use-case.  ^_^
>>> 
>>> More seriously, if the EME stuff is so foreign that it can't even
>>> integrate with the <video> source selection, that's a bug in the EME
>>> spec, not something to be worked around in other specs.
>> 
>> It's not the Encrypted Media Extensions, it's the Media Source Extensions
>> - there really is *nothing* associated with encrypted media in MSE (by
>> design) and many people would like to use one without the other.
>> 
>> The object of the Media Source Extensions is to enable adaptive streaming
>> to be implemented in Javascript. Rather than have the media element
>> select and download chunks of media, the Javascript application does that
>> (using XMLHttpRequest) and passes the downloaded chunks to the media
>> element using the Media Source Extensions.
>> 
>> Here is not really the place to debate the pros and cons of that
>> architecture, but there are a lot of advantages compared to the
>> alternative of embedding the adaptive streaming logic and manifest format
>> into the browser.
>> 
>> In this model, resource selection also becomes the job of the Javascript
>> application, which likely has a lot more information about the available
>> streams (as described in whatever manifest format the application uses)
>> to inform it's decisions. It just needs access to information about
>> codecs support (which it has through canPlayType) and output device
>> properties. Media queries seem the natural way to handle the latter,
>> since this is how that is already handled for the media element itself.
>> 
> Yes, but doesn't MSE allow you to attach yourself to a specific <source>?
> I 
> think what Tab is suggesting is that it may be better to let the media
> element
> pick the right <source> for the user (as we want media elements to do when
> there is no MSE involved). Then your JS knows which one is right
> implicitly 
> and grabs the right stream.
> 

There are two issues here:

First, the media element presently picks the source based on the type and media attributes. The media attribute allows to user to specify when a source should be used in terms of physical properties of the output device.

Tab suggests we need some way to declare the resolution, or channel count, of the media to the UA, but that would move the responsibility for deciding how to match media properties with physical properties from the user to the UA: a different distribution of responsibility from selection based on media queries. Why should these particular properties (physical resolution, audio output channels) be treated differently from others (color depth, aspect ratio, etc.) ?

I still think what is needed is media query properties for these properties of the output device.

Second, we could let the media element select between multiple <source> elements, each connected to a different MediaSource object. Each <source> would have a media query specifying when it should be used. The MediaSource associated with the chosen <source> would change state when selected, with an event indicating to the script that this has happened, prompting the script to start appending the appropriate media to this object. Everything needed to do this exists (except the additional properties described above).

The issue with this is combinatoric explosion. Suppose I have SD, HD and UHD video streams and stereo, 5.1 and 7.1 audio. I need to provide <source> elements with associated media queries. If I add 60fps as well as 30fps now I have 18. Ok, maybe a few less because I don't have the SD in 60fps…

Alternatively, I can call window.matchMedia() at most 8 times at which point I will know exactly which video and audio streams to use and can create a single MediaSource and append these. Again, though, everything needed to do this exists except the additional properties.

…Mark


>

Received on Wednesday, 20 March 2013 23:07:44 UTC