Re: Raw audio access (Re: Raw data API - 4 - direct RTP access)

> On 31 May 2018, at 09:49, Harald Alvestrand <harald@alvestrand.no> wrote:
> 
> Den 31. mai 2018 08:47, skrev westhawk:
>> I think I must have missed something - what Audio use-cases are we
>> addressing - that can’t be done with
>> the existing (or slightly extended) WebAudio API?
> 
> Changing subject, since this is not about the RTP API:
> 

I think that is where I got confused - what _is_ the RTP API for ?
I read lots of discussion about how to re-implement various layers in javascript,
but I’m not seeing a reason - or perhaps I simply missed it.


> For raw audio, the use cases are the ones where the overhead of
> converting the samples to Float32Array at 44.1k sample rate is
> unreasonable, but the overhead of converting to (for example) int16
> arrays at the incoming sample rate isn’t.

My instinct is that a _lot_ of raw audio use-cases could be addressed by
disabling various processing layers in the current stack and passing raw data
to webAudio - where one could implement codecs, PLC or echo cans or whatever.
This means extending the Streams/WebAudio interfaces a bit - but perhaps not so 
very much.

> 
> (WebAudio's sample rate is adjustable, but it's set for a whole
> AudioContext, not on each individual audio media flow).
> 

Maybe that means we need to be able to have multiple WebAudios
connected to a peer-connection.

So perhaps if you can create a WebAudio to speak directly to a G722 stream then you get
an 8khz (old joke) sample rate and a separate one to talk 48k stereo opus.

> 
> This is a pretty small set of applications, I think, so it may mean that
> raw audio API is pretty far down the list of APIs we feel like working on.
> 
> I'm trying to make sure we have all the pieces that people have been
> discussing up on the dartboard. I'd guess that after the Stockholm
> meeting, we'd decide that one or two of them is going to be chewed on
> first, a couple of others may be shelved for good, and a few more are
> going to be left in limbo (together with mediacapture-depth, for
> instance) until someone picks them back up.
Agree.

>

Received on Thursday, 31 May 2018 08:10:06 UTC