Re: Integer PCM sample formats to Web Audio API?

As per ROC's suggestion, that change would allow an implementation to
optimize by storing in the original format.  It's still going to have to do
the conversion in realtime, though.  And if the data of the buffer is
accessed, it would be in floating point (aka it would force a conversion of
the entire buffer).


On Wed, Jan 8, 2014 at 9:47 AM, Jukka Jylänki <jujjyl@gmail.com> wrote:

> Yes, that would definitely help with the int->float->int problem for
> mobile, and allow lifting JS code from having to implement signal format
> conversion. However it will not directly help with memory issues. If the
> browser stores the data internally as float, it will still require twice
> the memory. If that improvement would let browser store the audio data as
> 16-bit int as well, then I think it would be helpful.
>
>
> 2014/1/8 Chris Wilson <cwilso@google.com>
>
>> Jukka, you didn't reply specifically to ROC's suggestion:
>>
>> I think it would make sense to add an API to construct an AudioBuffer
>> from signed 16-bit PCM data. Then, as long as you don't call
>> getChannelData() on the buffer, the Web Audio implementation can probably
>> optimize its way to victory at least for the simple cases.
>>
>>
>> To be honest, I'm not sure that I can see doing much more than that
>> (effectively meaning run-time conversion from int to float) - the Web Audio
>> system - and many applications written for it - really do pretty much
>> require floating point in the pipeline, or clipping would destroy much of
>> the utility.  Do you see the above as a rational middle ground that we
>> should explore?
>>
>>
>> On Wed, Jan 8, 2014 at 7:21 AM, Jukka Jylänki <jujjyl@gmail.com> wrote:
>>
>>> Direct3D vs OpenGL is not the correct analogy here. If Web Audio spec
>>> resolved the need for integer sample formats as WONTFIX, it would be the
>>> equivalent of WebGL having said "For simplicity, we'll only do 4xFloat32
>>> RGBA textures as the only texture format type and don't support any other
>>> integer formats or compressed texture formats. That shall be the WebGL
>>> way.", which would have been completely horrible.
>>>
>>> Instead, OpenGL vs Direct3D is a very good example to the contrary, they
>>> are exactly in the feature parity competition business. Direct3D 11.2 just
>>> recently introduced support for "Tiled resources" (
>>> http://msdn.microsoft.com/en-us/library/windows/desktop/dn312084(v=vs.85).aspx), a way to do virtualized textures or "megatexturing". OpenGL done the
>>> same in OpenGL 4.4 with GL_ARB_sparse_texture (
>>> http://www.techpowerup.com/187584/khronos-releases-opengl-4-4-specification.html). Why? Since they understand that in order to stay viable as a seriously
>>> investable spec, they must compete in features. The way you program against
>>> the two APIs is immaterial in comparison, but that they both support the
>>> same _use cases_ and _features_ is extremely important. One should not
>>> confuse an API, i.e. the way you program against the features, and the
>>> features themself that the API offers.
>>>
>>> 2014/1/8 Marcus Geelnard <mage@opera.com>
>>>
>>>>  What I mean is that you should keep the Web platform in mind when
>>>> designing your game engine ...
>>>>
>>>
>>> My previous post tried to illustrate why this is a not a valid
>>> counterargument to make. For projects like HTML5 port of ScummVM:
>>> http://clb.demon.fi/html5scummvm/ or the Javascript MESS emulator:
>>> http://jsmess.textfiles.com there is no "designing your game engine"
>>> step.
>>>
>>> 2014/1/8 Marcus Geelnard <mage@opera.com>
>>>
>>>> otherwise you'll just be doing a port...
>>>>
>>>
>>> Yes! We are doing a port!
>>>
>>> 2014/1/8 Marcus Geelnard <mage@opera.com>
>>>
>>>> ... and it is quite likely that the port will perform noticeably worse
>>>> on the Web compared to the originally intended platform(s).
>>>>
>>>
>>> And this will happen exactly if the Web platform is rigid and was not
>>> designed to support the same features that the projects that are to be
>>> ported needed. But since we are in the roles that can have a say over what
>>> features Web Audio supports, it does not have to be that way.
>>>
>>>    Jukka
>>>
>>
>>
>