Re: Questioning the current direction of the Web Audio API

Ok let's rewind a bit. I **am** super excited myself about the idea behind
Web Audio. I know that you can - already now - implement super cool stuff
with it, and make them work really well. But what I am a bit sad about is
the whole architecture which make it almost impossible to extend it
properly.

First, for the people which are not convinced that extensibility should be
the number 1 priority here, let me repeat that sound production softwares
come bundled with a HUuuuge amount of effects.
Softwares like Propellerhead reason shipping with their own invented
synthesizers (https://www.youtube.com/watch?v=5nziQPZ4kTY), and softwares
like Ableton Live shipping with an impressive amount of high-quality
effects and plugins. Not mentioning VST plugins, without which the world of
DAWs wouldn't be what it is.
I know a lot of professional musicians, and trust me they would never work
with the couple of effects you could build with the current set of
AudioNodes.
And no matter if there is more nodes in the futures, there is just no way
all the basic building blocks for all the algorithms humans can ever
conceive can be provided as AudioNodes (and that sucks. Because on
basically every other plateform, there is no limitation).

Second, if you understand that professionals need things that can't be
built with basic AudioNodes, you understand that ScriptProcessorNode will
be more than just an escape valve. Now the big problem with that is : you
will need to instantiate multiple ScriptProcessorNodes in your graph,
connect them with native AudioNodes, and because of the sum of the
overheads of using ScriptProcessorNodes, you will end-up in a situation
where it is actually more performant to just put the whole dsp function
into ONE single ScriptProcessorNode, re-implementing oscillators,
convolutions, and the whole thing ... making native AudioNodes useless.
That's what I mean by "this architecture is impossible to extend".
I might be wrong (if I am please tell me so!), but if I am right, this
makes native AudioNodes rather pointless.

Cheers :)

Sebastien


2013/10/18 Joseph Berkovitz <joe@noteflight.com>

> Sebastien,
>
> I think you are really raising two different issues here, each of which is
> a good point that deserves its own response. Of course I don't speak for
> the many others in the WG but here's my take.
>
> First, ScriptProcessorNode was intended to work better than it apparently
> does. To the extent that this blocks the implementation of custom nodes,
> this is a situation that needs to be cleaned up quickly by this group
> because, indeed, ScriptProcessorNode was always supposed to be the "escape
> valve" for everything that wasn't a primitive node. However the flaw is not
> necessarily in the use of JS to script arbitrary operations; it may have
> more to do with the way that browser threads are used to perform these
> scripts.
>
> Second, the set of AudioNodes is not closed for all time. I think we all
> hope it will expand (for instance, we discussed a noise generator as an
> obvious gap that should be filled). The current set in Web Audio is of
> course limited by what the group and the browser vendors could fit into a
> v1 timeframe, and filtered according to the group's best take on priorities.
>
> The first issue obviously makes the second issue much, much worse and what
> we are hearing from both you and Hongchan is that it isn't usable in its
> current form. I feel the larger priority for the group should be on #1,
> making ScriptProcessorNode as robust as possible, given the effort involved
> in standardizing and adding new AudioNode flavors to the spec. Possibly the
> TAG recommendations are a starting point for doing this.
>
> Having said all this, there is still a great amount of room for creativity
> and experimentation and, yes, "generative music". We're a lot better off
> than we were before Web Audio, and I believe in the overall direction even
> if the present-day realization has flaws.
>
> Best,
>
> .            .       .    .  . ...Joe
>
> *Joe Berkovitz*
> President
>
> *Noteflight LLC*
> Boston, Mass.
> phone: +1 978 314 6271
> www.noteflight.com
> "Your music, everywhere"
>
> On Oct 18, 2013, at 11:25 AM, s p <sebpiq@gmail.com> wrote:
>
> @Olivier : if this is the crux of the matter, what is the idea behind the
> existence of AudioNodes?
>
> @Hongchan : I really didn't mean to criticize your work. In fact, this was
> not clear in the original post, but when I referred to "few good libraries
> out there", I was referring to waax as well ... which I have used, and
> which I got inspiration from to do the current round of WebPd refactoring.
> So all apologies if you misunderstood me.
> On the other hand, you cannot say that looping over a buffer of white
> noise to generate white noise is not an ugly hack ... and though you
> probably do your best to implement as many nodes as possible by composing
> the existing AudioNodes, you will never ever get even close to a library
> like SuperCollider in terms of amount and quality of UGens available.
>
>
> 2013/10/18 Hongchan Choi <choihongchan@gmail.com>
>
>> Hello All,
>>
>> I am Hongchan, the author of 'cruelly' lacking and 'ugly' WAAX. Since my
>> work is brought up in the topic, I guess I have to defend myself somehow.
>>
>> I went down the same path with OP at some point, that was simply because
>> I have been a computer musician myself over a decade. It is all about
>> experiments and I am very well aware of that.
>>
>>  First two revisions of the library were completely based on
>> ScriptProcessorNode - had to dump them all because they were not usable in
>> the real-world production. That was the moment I changed the goal and the
>> design; *something that runs without glitches.*
>>
>> Now I have been working on this API for a while (even with Chris Rogers
>> himself), I just can't say everything has been failed. I would say this API
>> is built for the production. On that note, it is just not as useful as PD,
>> SC, or ChucK for the experimental purposes.
>>
>> Currently I am refining the latest revision (r13) of WAAX<https://github.com/hoch/waax/tree/dev>and we (me and Chris Rogers) had put some ideas into it in order to
>> implement essential building blocks solely based on the native nodes by
>> utilizing Web Audio API in different ways. This is not public yet, and
>> hopefully I can wrap up the long-overdue documentation.
>>
>> *I am certain that there are many things we can achieve on top of the
>> current design of Web Audio API*. I found the majority of web audio
>> projects overlooks the countless possibility. Nonetheless, I cannot say OP
>> is wrong. I had the same complaints and rants once, but I just decided to
>> look at the other side.
>>
>> While I am at it, I would like to thank all the people in the audio
>> working group. This is a fantastic work!
>>
>> Best,
>> Hongchan
>>
>>
>>
>>
>> On Fri, Oct 18, 2013 at 7:01 AM, s p <sebpiq@gmail.com> wrote:
>>
>>> Answer from Chris Lowis :
>>>
>>>  Hi Sebastien, Thank you very much for getting in touch, it's great to
>>> hear from computer musicians and to learn more about your requirements.
>>> I'll reply in-line here, but perhaps we could continue the discussion as a
>>> group on public-audio@w3.org?
>>>
>>> > ry similar paradigm). It turned out to be pretty much impossible. For
>>> a simple reason is that Web Audio API really lacks objects, so I would have
>>> to implement most of them using **ScriptProcessorNodes**, and then loose
>>> all the benefits of using Web Audio API (all dsp in one ScriptProcessorNode
>>> would be faster).
>>>
>>> Could you clarify what you mean by "objects"? Do you mean node types,
>>> and in particular one-to-one mapping to existing nodes within PD - or are
>>> you talking about a JavaScript "object" layer on top of Web Audio?
>>>
>>> > The only stab - that I know of - at implementing some serious sound
>>> programming library on top of other WAA nodes is [waax](
>>> https://github.com/hoch/waax). But it cruelly lacks objects, and uses a
>>> couple of [ugly hacks](
>>> https://github.com/hoch/WAAX/blob/master/src/units/generators/Noise.js#L14
>>> ).
>>>
>>> I could do with a clarification of "objects" again here, just to help
>>> understand what you mean.
>>>
>>> > I love the idea of Web Audio API. But right now I feel that it really
>>> lacks prespective, and a clear direction.
>>>
>>> I think it's fair to say that the Web Audio API targets, at least in the
>>> initial "version 1" form common use cases on the web where previously one
>>> may have used Flash, plugins or hacks around the <audio> element. Having
>>> said that, there has been a large amount of interest from the computer
>>> music community in the API, and there is certainly a lot of interest in
>>> developing more in this direction.
>>>
>>> > I'd really like to hear people's opinion about why they do it like
>>> that, how and why they think it can/will be used for real-life
>>> applications, because the goals stated in the draft are - in my humble
>>> opinion - completely unrealistic with the current functionalities.
>>>
>>> Our Use Cases document gives a good idea of the kind of real-life
>>> applications we are targetting:
>>> https://dvcs.w3.org/hg/audio/raw-file/tip/reqs/Overview.html
>>>
>>> > I am sorry to be a bit harsh, and question this project in its
>>> foundations, but I suppose that's what you get for being implied in open
>>> standards : any random angry guy out there can come and complain :)
>>>
>>> Not at all, speaking personally I think what you are doing is
>>> fascinating and something I hope more people will attempt using the API in
>>> the future. Please keep the discussion going! Cheers, Chris
>>>
>>
>>
>>
>> --
>> Hongchan Choi
>>
>> PhD Candidate, Research Assistant
>> Center for Computer Research in Music and Acoustics (CCRMA)
>> Stanford University
>>
>> http://ccrma.stanford.edu/~hongchan
>>
>
>
>
> --
> *Sébastien Piquemal
> *
> *
> ** *-----* @sebpiq*
>  -----* *http://github.com/sebpiq*
> *
>  ----- http://funktion.fm
>
>
>
>


-- 
*Sébastien Piquemal
*
*
** *-----* @sebpiq*
 -----* *http://github.com/sebpiq*
*
 ----- http://funktion.fm