[mediacapture-output] Controlling 3rd party iframe audio output on a page?

randallb has just created a new issue for 
https://github.com/w3c/mediacapture-output:

== Controlling 3rd party iframe audio output on a page? ==
(forgive me, this is my first time attempting to contribute)

We've built an application to allow for video streaming, and we 
essentially render the video using a browser, then output the 
framebuffer to another encoding process and send it along to an RTMP 
destination. (In our case, it's usually FB Live.)

We'd like to enable our users to easily embed other media in their 
productions.

Today, we capture the audio by changing a machine's default audio 
device to one we custom wrote, then reroute the audio to our encoder 
process. This works great generally, but has the major drawback of 
disabling audio for the rest of the computer.

As the web audio spec has started shipping in Chrome specifically, 
we've started experimenting with using the web audio output api to 
redirect the audio properly. Basically, we use the enumerate devices 
api to find our driver, and if a confluence of things are correct, we 
direct our audio to go out to that spot explicitly using the setSinkId
 of audio and video elements.

The issue is if we'd like to embed other external media, like an 
iframe from YouTube as a simple example, we'd need YouTube to 
explicitly support switching audio destinations in their postmessage 
api. We view this as unlikely given our usecase is more edgecase for 
their business. We think the top-most context for a page should likely
 be in charge of where audio ends up, if inner iframes haven't changed
 their sound settings past 'default'. Basically, the top-most context 
could be in charge of all audio routing ideally.

I'd propose a `setSinkId` api on an iframe, just like we have on audio
 / video elements. If this has been done before, I apologize, I wasn't
 able to find any data on this pretty much anywhere on the web.

I think there's likely some technical challenges here, but I think for
 advanced audio / video (what i'm obsessed with) it'll help a lot with
 what the web is great at: linking and embedding resources.

Please view or discuss this issue at 
https://github.com/w3c/mediacapture-output/issues/63 using your GitHub
 account

Received on Saturday, 5 November 2016 12:12:39 UTC