Re: active speaker information in mixed streams

On Tue, Jan 28, 2014 at 2:50 PM, Emil Ivov <emcho@jitsi.org> wrote:
> On Tue, Jan 28, 2014 at 11:40 PM, Peter Thatcher <pthatcher@google.com> wrote:
>> Ah, I guess I was assuming your mixer was under your control and you
>> could make it speak the data channel.
>
> It is, but then there's no reason for it to speak SCTP. And besides
> ... datachannel support in a mixer is ... what exactly? Data passes
> through and then everything goes to everyone? This would probably
> break more cases than it would help solve. Or should the mixer also be
> a WebRTC endpoint? What does that even mean for  a mixer? What
> protocol does it handle to negotiate data channels?

Sounds like a reasonable idea to me.  Why not use Jingle?

>
>> Your use case, then, is more that you have a legacy endpoint (which
>> happens to be a mixer) sending certain RTP,
>
> Well no not really. It does ICE and DTLS/SRTP so I wouldn't call it
> legacy. It does try to keep to its own layer of the stack though and
> it would rather not be an application-specific network entity with
> baked-in application layer signalling.
>
>> and you want to be able to
>> extract certain info from the RTP and have it available in the JS?  Is
>> that correct?  If so, what is the exact information you want to
>> extract an have available?
>
> Sorry I thought I had mentioned that already. That would be CSRCs or
> audio level header extensions as per RFC6465.

Sorry, I got distracted by the mixer stuff.  Let's forget about that
distraction and just focus on the RFC6465 and CSRC bits.

In that case, I'm with Harald: it probably won't fit into 1.0.  Hope for 2.0 :).


>
> Emil
>
> --
> https://jitsi.org
>
>>
>> On Tue, Jan 28, 2014 at 2:30 PM, Emil Ivov <emcho@jitsi.org> wrote:
>>> Hey Tim, Peter,
>>>
>>> Sync isn't really the issue here. It's mostly about the fact that the
>>> mixer is not a WebRTC entity. This means that it most likely doesn't
>>> even know what SCTP is, it doesn't necessarily have access to
>>> signalling and above all, the mix is likely to also contain audio from
>>> non-webrtc endpoints. Using DataChannels in such situations would
>>> likely turn out to be quite convoluted.
>>>
>>> Emil
>>>
>>> On Tue, Jan 28, 2014 at 9:39 PM, tim panton <thp@westhawk.co.uk> wrote:
>>>>
>>>> On 28 Jan 2014, at 13:28, Emil Ivov <emcho@jitsi.org> wrote:
>>>>
>>>>> Hey all,
>>>>>
>>>>> One requirement that we often bump against is the possibility to
>>>>> extract active speaker information from an incoming *mixed* audio
>>>>> stream. Acquiring the CSRC list from RTP would be a good solution to
>>>>> this. Audio levels as per RFC6465 would be even better.
>>>>>
>>>>> Are these something that could be foreseeable for 1.0 ?
>>>>>
>>>>
>>>> Would it be possible to extract the relevant audio levels with the WebAudioApi and
>>>> then send the data points over the DataChannel. I realise the sync won't be exact,
>>>> but might it do the trick ?
>>>>
>>>>> Emil
>>>>>
>>>>> --
>>>>> https://jitsi.org
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Emil Ivov, Ph.D.                       67000 Strasbourg,
>>> Project Lead                           France
>>> Jitsi
>>> emcho@jitsi.org                        PHONE: +33.1.77.62.43.30
>>> https://jitsi.org                       FAX:   +33.1.77.62.47.31
>>>
>
>
>
> --
> Emil Ivov, Ph.D.                       67000 Strasbourg,
> Project Lead                           France
> Jitsi
> emcho@jitsi.org                        PHONE: +33.1.77.62.43.30
> https://jitsi.org                       FAX:   +33.1.77.62.47.31

Received on Tuesday, 28 January 2014 23:17:23 UTC