Re: Music Synthesis

Apple has a great optimized vector library called vecLib.framework.  I used
it whenever possible when I was on the Core Audio team, and I've also made
use of it when appropriate in my WebKit implementation.  But audio
processing involves a great deal more than simple calls to vector libraries.
 Most DSP algorithms are more complex than that and carry a lot of state
information around.  I had to hand-optimize lots of my custom code
(initially with altivec, then later with SSE).  I know that the Core Audio
team continues to spend lots of time optimizing different components of
their audio stack, especially now for mobile devices.  It really is a lot
more than a series of calls to vector library functions.

Also, I believe it's very important to provide a useful, high-level
javascript API that's dependable, scalable, and powerful.  Simply providing
a library of vector calls, and then hoping that javascript will be fast
enough to take up the slack in a worker thread just doesn't seem realistic.
 It would require all browser vendors to implement worker threads and
javascript runtimes with real-time constraints in mind to guarantee good
latency performance and glitch-free operation.  This is extremely difficult
and is a lot to expect from all the different browser vendors.  I believe
that not executing this properly would result in an abysmal user experience
and web developers would not trust that their audio applications could be
deployed across the spectrum of browsers and devices.

Chris


On Wed, Jun 16, 2010 at 3:30 PM, Robert O'Callahan <robert@ocallahan.org>wrote:

> On Thu, Jun 17, 2010 at 3:36 AM, Chris Marrin <cmarrin@apple.com> wrote:
>
>> Chris' proposal is node based. As such, it can easily be optimized for a
>> given platform. The Mozilla proposal simply provides audio samples which can
>> be used by JavaScript and then played (or not in the case of audio
>> visualizers). It sounds like you're advocating an API to go with the Mozilla
>> proposal. That can be optimized too, but it's not currently being proposed.
>> If it were, it would be an SVG (Chris' proposal) vs Canvas (Mozilla's
>> proposal with an API) discussion. Either way, I don't believe access to the
>> audio samples with pure JavaScript processing is sufficient.
>>
>
> So if we had an WebGLArray (or whatever it gets renamed to in ES5)
> processing library amenable to parallel/vector/GPU optimization, and a way
> of delivering samples to a Worker for processing to avoid main-thread
> latency, do you have any fundamental objection to that kind of approach?
>
> Rob
> --
> "He was pierced for our transgressions, he was crushed for our iniquities;
> the punishment that brought us peace was upon him, and by his wounds we are
> healed. We all, like sheep, have gone astray, each of us has turned to his
> own way; and the LORD has laid on him the iniquity of us all." [Isaiah
> 53:5-6]
>