Re: Audio Workers - please review

On Fri, Sep 12, 2014 at 4:30 PM, Chris Wilson <cwilso@google.com> wrote:

> I'm probably going to sound like a jerk in this response, so I'll
> apologize in advance; I've thought through parallelism scenarios a lot as a
> prelude to the audio worker proposal, and been left with a very strong
> "best left to the developer making informed decisions" feeling.
>
> Are you presuming that the system would auto-analyze to find optimal
> subgraphs, and then the user just gets latency added to that sub-graph, or
> you would try to "work ahead" in the subtree to make up for the inherent
> inter-thread communication latency?  How frequently would you analyze to
> make that decision and would you inform the developer somehow?  Would you
> let them override?  Would any streaming (live input, media element sources)
> disable parallelism, then?
>
> I get that it's possible to analyze a subgraph and decide it's 1)
> computationally expensive and 2) not interconnected, and thus might be a
> good candidate.  But I think inserting latency at ANY point in the graph
> connections, NOT at the explicit request of the developer, is a bad idea.
>  If you inserted a 50ms delay in one side of a graph that's supposed to
> play at the same time as another subgraph that isn't moved into another
> thread, that is going to have bad side effects.  And given the standard
> shape of most mixer graphs (e.g. the standard channel strip/sends model),
> the computationally expensive bits ARE going to be interconnected.
>
> If you say the point here is to enable automatic parallelism with zero
> additional latency (by only parallelizing in situations where you can work
> ahead in time and get it to align properly), then I think that's possible,
> but a big challenge, relatively low return on investment, and much lower
> priority than 99% of the things I feel we have left on our plate.
>

Given that for example in C++ you can block a thread until a given task in
another thread is finished, parallelizing parts of the graph doesn't
necessarily mean that they have to be rendered ahead of time or introduce
latency to the graph. IIRC for example the original implementation of the
ConvolverNode actually parallelized its work and blocked the audio thread
until the work was finished. That's not to say that the parallelization
comes for free, it never does (deadlocks, anyone?), but I don't think we
should make choices that exclude that option. Not that we are making any
that I know of, though.


> If you're implying auto-parallelization that introduces observable
> latency, I think that is a bad idea.
>
> On Fri, Sep 12, 2014 at 3:11 PM, Robert O'Callahan <robert@ocallahan.org>
> wrote:
>
>> On Sat, Sep 13, 2014 at 1:02 AM, Chris Wilson <cwilso@google.com> wrote:
>>
>>> 1) I do not believe arbitrary parallelization of the graph is a good
>>> idea. It will be moderately difficult to examine the graph to decide that
>>> it's "okay" to parallelize (i.e. that there are no interconnections or
>>> other dependencies), and far worse, the process will needfully insert
>>> latency into the graph.  I believe if authors think they should parallelize
>>> their app, they can do that (with workers and inserting their own latency)
>>> - I've discussed this with a couple of pro audio app builders, and they
>>> were comfortable with this.  (that is to say: you don't get arbitrary
>>> parallelization as optimization in any other system I know of.  It has side
>>> effects.)
>>>
>>
>> A parallelism analysis could be done in, er, parallel with the actual
>> audio processing.
>>
>> I'm reasonably optimistic that auto-parallelization of Web Audio graphs
>> could pay off.
>>
>> Rob
>> --
>> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
>> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
>> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
>> owohooo
>> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
>> oioso
>> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
>> owohooo
>> osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
>> ooofo
>> otohoeo ofoioroeo ooofo ohoeololo.
>>
>
>