W3C home > Mailing lists > Public > public-webrtc@w3.org > May 2018

Re: Raw audio access (Re: Raw data API - 4 - direct RTP access)

From: westhawk <thp@westhawk.co.uk>
Date: Thu, 31 May 2018 10:09:40 +0200
Cc: public-webrtc@w3.org
Message-Id: <26CF5723-CA77-49EA-BF3C-8471C8D3EE0D@westhawk.co.uk>
To: Harald Alvestrand <harald@alvestrand.no>


> On 31 May 2018, at 09:49, Harald Alvestrand <harald@alvestrand.no> wrote:
> 
> Den 31. mai 2018 08:47, skrev westhawk:
>> I think I must have missed something - what Audio use-cases are we
>> addressing - that can’t be done with
>> the existing (or slightly extended) WebAudio API?
> 
> Changing subject, since this is not about the RTP API:
> 

I think that is where I got confused - what _is_ the RTP API for ?
I read lots of discussion about how to re-implement various layers in javascript,
but I’m not seeing a reason - or perhaps I simply missed it.


> For raw audio, the use cases are the ones where the overhead of
> converting the samples to Float32Array at 44.1k sample rate is
> unreasonable, but the overhead of converting to (for example) int16
> arrays at the incoming sample rate isn’t.

My instinct is that a _lot_ of raw audio use-cases could be addressed by
disabling various processing layers in the current stack and passing raw data
to webAudio - where one could implement codecs, PLC or echo cans or whatever.
This means extending the Streams/WebAudio interfaces a bit - but perhaps not so 
very much.

> 
> (WebAudio's sample rate is adjustable, but it's set for a whole
> AudioContext, not on each individual audio media flow).
> 

Maybe that means we need to be able to have multiple WebAudios
connected to a peer-connection.

So perhaps if you can create a WebAudio to speak directly to a G722 stream then you get
an 8khz (old joke) sample rate and a separate one to talk 48k stereo opus.

> 
> This is a pretty small set of applications, I think, so it may mean that
> raw audio API is pretty far down the list of APIs we feel like working on.
> 
> I'm trying to make sure we have all the pieces that people have been
> discussing up on the dartboard. I'd guess that after the Stockholm
> meeting, we'd decide that one or two of them is going to be chewed on
> first, a couple of others may be shelved for good, and a few more are
> going to be left in limbo (together with mediacapture-depth, for
> instance) until someone picks them back up.
Agree.

> 
Received on Thursday, 31 May 2018 08:10:06 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:18:41 UTC