Re: Integer PCM sample formats to Web Audio API?

We don't actually even NEED to add anything to the API, we could simply
mention it's a possible optimization to implementations to choose to keep
integer-based representations in their original form until getChannelData
is called.


On Fri, Jan 10, 2014 at 12:53 AM, Marcus Geelnard <mage@opera.com> wrote:

> 2014-01-09 20:49, Paul Adenot skrev:
>
>     - A way to query whether a given buffer is int16 or float32, to allow
>>>> code to assert() that it is getting what it desires. Otherwise debugging
>>>> can be a pain if the only way is for the programmer to measure this in
>>>> indirect ways via looking at memory consumption in a system process
>>>> manager.
>>>>
>>> I would NOT want to do this.  This should be under the hood, and some
>>> systems (e.g. desktops with lots of memory) may choose to always expand.
>>>
>> I'd rather do something explicit, yes, but see below.
>>
>
> I would like to avoid possibilities of inspecting engine internals like
> this. It raises the risk of site breakage and limits the ways in which an
> implementation can do optimizations.
>
>
>
>
>>     - If the support for int16 is specced in to be optionally supportable
>>>> (I
>>>> hope this will not be the case!), a way to ask the implementation if it
>>>> does support int16 format.
>>>>
>>>>  I don't think you'd need this.  It's going to end up being a
>>> performance
>>> tradeoff - memory for CPU - that's likely better made by the platform
>>> implementation than the developer.  If we did want this, I'd expect this
>>> is
>>> a different decodeAudioData-style call.
>>>
>> I think that would address the use-case, here, while being
>> retro-compatible.
>>
>> We could pass in an optional value as last argument of decodeAudioData,
>> that would indicates that the author would rather save memory than have
>> a good quality / save some CPU.
>>
>
> I agree with Chris here. In general, an implementation has a much better
> chance of making an informed performance trade off decision than a Web
> developer. In most situations I think that int16 would save memory, but add
> some extra CPU overhead. In other words, if a client has loads and loads of
> memory, it could for instance decide to take the memory hit and convert to
> Float32 internally in order to lower the CPU load, either up front or
> lazily.
>
> If a Web page was to make the trade off decision, it would have to know a
> lot more about the device it's running on (such as CPU performance,
> available memory, etc), and since that is not the case, I fear that Web
> devs would resort to things like UA sniffing etc to make distinctions such
> as "desktop" vs "fast-tablet" vs "slow-phone", etc.
>
>
>
>> It seems that it could be useful to specify sample rate and bit depth in
>> the decoded buffer, game developers have told us in the past that they
>> frequently use 22050Hz (or less) sounds in games, as well as 8/16 bits
>> buffers.
>> Knowing their assets, only them can make the good call between cutting on
>> the
>> sample rate, the bit depth, or both.
>>
>
>
> This is a slightly different issue, namely "What's the lowest quality I
> can accept for this asset?". I can see a value in giving the Web dev the
> possibility to indicate that a given asset can use a low quality internal
> representation (the exact syntax for this has to be worked out, of course).
> The situation is somewhat similar to how 3D APIs allow developers to use
> compressed textures when a slight quality degradation can be accepted. For
> audio, I think that sounds such as noisy or muddy sound effects could
> definitely use a lower quality internal representation in many situations.
> The same could go for emulators that mainly use 8/16-bit low-sample-rate
> sounds.
>
> Having such an option in the API gives the implementation an opportunity
> to save memory when memory is scarce, but it's not necessarily forced to do
> so.
>
> /Marcus
>
>
> --
> Marcus Geelnard
> Technical Lead, Mobile Infrastructure
> Opera Software
>
>