Re: active speaker information in mixed streams

On Tue, Jan 28, 2014 at 11:40 PM, Peter Thatcher <pthatcher@google.com> wrote:
> Ah, I guess I was assuming your mixer was under your control and you
> could make it speak the data channel.

It is, but then there's no reason for it to speak SCTP. And besides
... datachannel support in a mixer is ... what exactly? Data passes
through and then everything goes to everyone? This would probably
break more cases than it would help solve. Or should the mixer also be
a WebRTC endpoint? What does that even mean for  a mixer? What
protocol does it handle to negotiate data channels?

> Your use case, then, is more that you have a legacy endpoint (which
> happens to be a mixer) sending certain RTP,

Well no not really. It does ICE and DTLS/SRTP so I wouldn't call it
legacy. It does try to keep to its own layer of the stack though and
it would rather not be an application-specific network entity with
baked-in application layer signalling.

> and you want to be able to
> extract certain info from the RTP and have it available in the JS?  Is
> that correct?  If so, what is the exact information you want to
> extract an have available?

Sorry I thought I had mentioned that already. That would be CSRCs or
audio level header extensions as per RFC6465.

Emil

--
https://jitsi.org

>
> On Tue, Jan 28, 2014 at 2:30 PM, Emil Ivov <emcho@jitsi.org> wrote:
>> Hey Tim, Peter,
>>
>> Sync isn't really the issue here. It's mostly about the fact that the
>> mixer is not a WebRTC entity. This means that it most likely doesn't
>> even know what SCTP is, it doesn't necessarily have access to
>> signalling and above all, the mix is likely to also contain audio from
>> non-webrtc endpoints. Using DataChannels in such situations would
>> likely turn out to be quite convoluted.
>>
>> Emil
>>
>> On Tue, Jan 28, 2014 at 9:39 PM, tim panton <thp@westhawk.co.uk> wrote:
>>>
>>> On 28 Jan 2014, at 13:28, Emil Ivov <emcho@jitsi.org> wrote:
>>>
>>>> Hey all,
>>>>
>>>> One requirement that we often bump against is the possibility to
>>>> extract active speaker information from an incoming *mixed* audio
>>>> stream. Acquiring the CSRC list from RTP would be a good solution to
>>>> this. Audio levels as per RFC6465 would be even better.
>>>>
>>>> Are these something that could be foreseeable for 1.0 ?
>>>>
>>>
>>> Would it be possible to extract the relevant audio levels with the WebAudioApi and
>>> then send the data points over the DataChannel. I realise the sync won't be exact,
>>> but might it do the trick ?
>>>
>>>> Emil
>>>>
>>>> --
>>>> https://jitsi.org
>>>>
>>>
>>
>>
>>
>> --
>> Emil Ivov, Ph.D.                       67000 Strasbourg,
>> Project Lead                           France
>> Jitsi
>> emcho@jitsi.org                        PHONE: +33.1.77.62.43.30
>> https://jitsi.org                       FAX:   +33.1.77.62.47.31
>>



-- 
Emil Ivov, Ph.D.                       67000 Strasbourg,
Project Lead                           France
Jitsi
emcho@jitsi.org                        PHONE: +33.1.77.62.43.30
https://jitsi.org                       FAX:   +33.1.77.62.47.31

Received on Tuesday, 28 January 2014 22:51:27 UTC