Re: Integer PCM sample formats to Web Audio API?

2014/1/17, Chris Wilson <cwilso@google.com>:
> On Fri, Jan 17, 2014 at 2:24 AM, Marcus Geelnard <mage@opera.com> wrote:
>
>> So, when discussing Float32 vs Int16 etc, please keep in mind the use
>> cases where an AudioBuffer is used for accessing and possibly also
>> modifying audio data by using the getChannelData method on the
>> AudioBuffer,
>> such as:
>>
>> * ScriptProcessorNode / AudioProcessingEvent
>>
>
> I believe there's already a suggestion on the table to replace AudioBuffer
> there with Float32Array.

I'm all for that. I think it would be natural to consider that option
when specing the new worker-based script processor.

>
> There has already been a suggestion brought forward by ROC (i.e. allow the
>> use of Int16 internally), that should solve the most urgent memory
>> issues.
>> If that suggestion does not solve the problems at hand, please provide
>> more
>> information.
>>
>
> +1.  I'd still like to better understand the conversion impact.
>

If I can find the time I'll try and make some kind of benchmark of a
simple int16->float32 format converter.

> The open questions, to me, are 1) how does the data get EXPOSED then (i.e.
> does getChannelData still return a float32array, and force conversion),

I would prefer to keep it as Float32, at least for now. I see little
value in handing over integers to any kind of JS processing. The
implication would probably be that if you use getChannelData, you'll
force a conversion of the internal format to Float32.

> 2)
> if it is exposed in int16 or similar, how far down that rabbit hole do we
> go (int8, int24?, int32), and

IMO the added value of such an addition would not justify the API
complexity cost, plus it could easily be a slippery slope.

> 3) I will point out again that the 2x bloat
> from converting to int16 to float32 is potentially much less of a problem
> than the sample rate resampling (loading a 22kHz sample into a 96kHz audio
> context would cause a >4x bloat).
>

+1   It may be slightly trickier to drop the resampling step though,
since it could come with a quality penalty. I suggest that we give
that issue some attention.

Do we want to make it possible to opt out from the automatic resampling step?

/Marcus

Received on Friday, 17 January 2014 21:31:43 UTC