- From: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
- Date: Thu, 29 Mar 2012 08:14:47 +0200
- To: Chris Rogers <crogers@google.com>
- CC: "Wei, James" <james.wei@intel.com>, "public-audio@w3.org" <public-audio@w3.org>, "public-webrtc@w3.org" <public-webrtc@w3.org>
Hi Chris, thanks for quick feedback. I would just want to point out that my feedback was my personal, not for the webrtc WG. I hope we can come with WG feedback later on. Stefan On 03/29/2012 04:33 AM, Chris Rogers wrote: > Hi Stefan, thanks for you comments. I'll try to give some answers > inline below. > > For reference, please refer to this provisional document describing > scenarios for WebRTC / WebAudio interaction. > This is the same one that I posted to the list earlier last year and > presented to the WebRTC working group at TPAC 2011: > https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html > > As we discussed at TPAC, this document would need some small changes to > be able to handle MediaStreamTrack objects explicitly instead of > MediaStreams. Please note that these extensions to the Web Audio API > (from the webrtc-integration document above) are not *yet* in the main > Web Audio API specification document. I would like some help from the > WebRTC group to help finish this part of the API (from > the webrtc-integration.html document above). > > Please see rest of comments inline below. > > Thanks, > Chris > > On Wed, Mar 28, 2012 at 7:05 PM, Wei, James <james.wei@intel.com > <mailto:james.wei@intel.com>> wrote: > > From: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com > <mailto:stefan.lk.hakansson@ericsson.com>> ____ > > Date: Wed, 28 Mar 2012 20:54:38 +0200____ > > Message-ID: <4F735E6E.7070706@ericsson.com > <mailto:4F735E6E.7070706@ericsson.com>> ____ > > To: "public-webrtc@w3.org <mailto:public-webrtc@w3.org>" > <public-webrtc@w3.org <mailto:public-webrtc@w3.org>> ____ > > I've spent some time looking at the Web Audio API [1] since the > Audio WG ____ > > have put out a call for review [2].____ > > __ __ > > As starting point I used the related reqs from our use-case and req ____ > > document [3]:____ > > __ __ > > F13 The browser MUST be able to apply spatialization____ > > effects to audio streams.____ > > __ __ > > F14 The browser MUST be able to measure the level____ > > in audio streams.____ > > F15 The browser MUST be able to change the level____ > > in audio streams.____ > > __ __ > > with the accompanying API reqs:____ > > __ __ > > A13 The Web API MUST provide means for the web____ > > application to apply spatialization effects to____ > > audio streams.____ > > A14 The Web API MUST provide means for the web____ > > application to detect the level in audio____ > > streams.____ > > A15 The Web API MUST provide means for the web____ > > application to adjust the level in audio____ > > streams.____ > > A16 The Web API MUST provide means for the web____ > > application to mix audio streams.____ > > __ __ > > Looking at the Web Audio API, and combining it with the MediaStream ____ > > concept we use, I come to the following understanding:____ > > __ __ > > 1) To make the audio track(s) of MediaStream(s) available to the Web > ____ > > Audio processing blocks, the The MediaElementAudioSourceNode > Interface ____ > > would be used. > > > Generally, it would be a "MediaStreamAudioSourceNode" (or perhaps > "MediaStreamTrackAudioSourceNode"), please see example 3: > https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html > > ____ > > __ __ > > 2) Once that is done, the audio is available to the Web Audio API ____ > > toolbox, and anything we have requirements on can be done____ > > __ __ > > 3) When the processing has been done (panning, measure level, change > ____ > > level, mix) the audio would be played using an AudioDestinationNode ____ > > Interface > > > The AudioDestinationNode represents the local client's "speakers", in > example 3 a MediaStream is created with the line: > var peer = context.createMediaStreamDestination(); > > to send the processed audio to the remote peer. > > ____ > > __ __ > > What is unclear to me at present, is how synchronization would work. > So ____ > > far we have been discussing in terms of that all tracks in a > MediaStream ____ > > are kept in sync; but what happens when the audio tracks are routed > to ____ > > another set of tools, and not played in the same (video) element as > the ____ > > video? > > > HTMLMediaElements already have a mechanism for synchronization using the > HTML5 MediaController API. Live stream (live camera/mic or remote > peers) MediaStreams would maintain synchronization (I assume you mean > audio/video sync in this case). The Web Audio API would just be used to > apply effects, not changing the synchronization. > > ____ > > __ __ > > Another take away is that the processing can only happen in the > browser ____ > > that is going to play the audio, since there is no way to go from an > ____ > > AudioNode to a MediaStream or MediaStreamTrack. > > > There are some examples showing how to go from an AudioNode to > MediaStream (needs to be developed for MediaStreamTrack). > Please look especially at the use of the createMediaStreamSource() > and createMediaStreamDestination() methods. > https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html > > ____ > > __ __ > > Anyone else that has looked into the Web Audio API? And any other ____ > > conclusions?____ > > __ __ > > I think we should give feedback from this WG (as we have some reqs > that ____ > > are relevant).____ > > __ __ > > Br,____ > > Stefan____ > > __ __ > > __ __ > > [1] http://www.w3.org/TR/2012/WD-webaudio-20120315/ ____ > > [2] > http://lists.w3.org/Archives/Public/public-webrtc/2012Mar/0072.html ____ > > [3] ____ > > http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements/?include_text=1 > ____ > > __ __ > > Best Regards ____ > > __ __ > > James ____ > > __ __ > > __ __ > >
Received on Thursday, 29 March 2012 06:15:20 UTC