Re: Integer PCM sample formats to Web Audio API?

2014-01-09 20:49, Paul Adenot skrev:
>>>    - A way to query whether a given buffer is int16 or float32, to allow
>>> code to assert() that it is getting what it desires. Otherwise debugging
>>> can be a pain if the only way is for the programmer to measure this in
>>> indirect ways via looking at memory consumption in a system process manager.
>> I would NOT want to do this.  This should be under the hood, and some
>> systems (e.g. desktops with lots of memory) may choose to always expand.
> I'd rather do something explicit, yes, but see below.

I would like to avoid possibilities of inspecting engine internals like 
this. It raises the risk of site breakage and limits the ways in which 
an implementation can do optimizations.


>
>>>    - If the support for int16 is specced in to be optionally supportable (I
>>> hope this will not be the case!), a way to ask the implementation if it
>>> does support int16 format.
>>>
>> I don't think you'd need this.  It's going to end up being a performance
>> tradeoff - memory for CPU - that's likely better made by the platform
>> implementation than the developer.  If we did want this, I'd expect this is
>> a different decodeAudioData-style call.
> I think that would address the use-case, here, while being retro-compatible.
>
> We could pass in an optional value as last argument of decodeAudioData,
> that would indicates that the author would rather save memory than have
> a good quality / save some CPU.

I agree with Chris here. In general, an implementation has a much better 
chance of making an informed performance trade off decision than a Web 
developer. In most situations I think that int16 would save memory, but 
add some extra CPU overhead. In other words, if a client has loads and 
loads of memory, it could for instance decide to take the memory hit and 
convert to Float32 internally in order to lower the CPU load, either up 
front or lazily.

If a Web page was to make the trade off decision, it would have to know 
a lot more about the device it's running on (such as CPU performance, 
available memory, etc), and since that is not the case, I fear that Web 
devs would resort to things like UA sniffing etc to make distinctions 
such as "desktop" vs "fast-tablet" vs "slow-phone", etc.

>
> It seems that it could be useful to specify sample rate and bit depth in
> the decoded buffer, game developers have told us in the past that they
> frequently use 22050Hz (or less) sounds in games, as well as 8/16 bits buffers.
> Knowing their assets, only them can make the good call between cutting on the
> sample rate, the bit depth, or both.


This is a slightly different issue, namely "What's the lowest quality I 
can accept for this asset?". I can see a value in giving the Web dev the 
possibility to indicate that a given asset can use a low quality 
internal representation (the exact syntax for this has to be worked out, 
of course). The situation is somewhat similar to how 3D APIs allow 
developers to use compressed textures when a slight quality degradation 
can be accepted. For audio, I think that sounds such as noisy or muddy 
sound effects could definitely use a lower quality internal 
representation in many situations. The same could go for emulators that 
mainly use 8/16-bit low-sample-rate sounds.

Having such an option in the API gives the implementation an opportunity 
to save memory when memory is scarce, but it's not necessarily forced to 
do so.

/Marcus

-- 
Marcus Geelnard
Technical Lead, Mobile Infrastructure
Opera Software

Received on Friday, 10 January 2014 08:53:44 UTC