W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2012

Re: Help needed with a sync-problem

From: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
Date: Fri, 3 Aug 2012 20:11:50 +0300
Message-ID: <CAJhzemX4nx8S9_MNfOv77GHsEGUrW1L7gOYpws6Q-6dpaGj5wg@mail.gmail.com>
To: Peter van der Noord <peterdunord@gmail.com>
Cc: Adam Goode <agoode@google.com>, public-audio@w3.org
It's a known and major issue all right, but it's not a bug. There's not
much that can be done about it though, afaict. The processing thread has to
buffer enough data (the buffer size) from the inputs of the JSNode before
its callback can be invoked, and next it just sends an event to the JS
thread to process the buffer. The audio thread, however, can't wait for the
JS thread to process the buffer but instead plays back the previously
processed buffer.

This is one of the reasons why I think we should focus on empowering the JS
node / DSP API. If you want to add any custom processing to the graph,
you're going to have to adjust the rest of the graph accordingly and you'll
end up with more latency. This means that if you want to do extensive
custom processing you'll probably need to work around the graph or just go
with just the JS and have your own routing which means you're in a much
more flexible place already anyway. I think the graph serves best as an IO
abstraction, and that's the part we should focus on.

Cheers,
Jussi

On Fri, Aug 3, 2012 at 4:16 PM, Peter van der Noord
<peterdunord@gmail.com>wrote:

> Well, it seems indeed that custom-nodes add a delay-time to the signal.
> I've connected a few bypass modules (they write their input to the output)
> and i'm magically creating an echo...
>
> http://www.petervandernoord.nl/patchwork_js/?patch=2&buffer_size=8192
>
> (to hear sound, you have to select the loaded buffer from the pulldown in
> the buffersource-module)
>
> I'm getting somewhat confused and concerned about this, why does this
> happen and isn't this a major bug/issue?
>
> Peter
>
>
>
> 2012/8/2 Adam Goode <agoode@google.com>
>
>> I think you can use playbackTime to determine the absolute a-rate time of
>> the beginning of the javascript buffer. But last I checked it wasn't
>> present in webkit.
>>
>> You might be able to count samples, assuming you know the node's noteOn
>> time, to keep track of the a-rate time. But with a short buffer size,
>> sometimes you can have problems as you've noticed.
>>
>>
>> On Thu, Aug 2, 2012 at 4:47 PM, Peter van der Noord <
>> peterdunord@gmail.com> wrote:
>>
>>> Ermmm.....wait, what? And that is intened behavior?
>>>
>>> Peter
>>>
>>>
>>> 2012/8/2 Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
>>>
>>>> Hey Peter!
>>>>
>>>> I think this is because the JSNode has a delay equivalent to the buffer
>>>> size, hence if you have parallel graphs that contain a different number of
>>>> JSNodes, they will arrive to the common destination at a different delay.
>>>>
>>>> Cheers,
>>>> Jussi
>>>>
>>>>
>>>> On Thu, Aug 2, 2012 at 11:35 PM, Peter van der Noord <
>>>> peterdunord@gmail.com> wrote:
>>>>
>>>>> I'm having a strange problem with some signals at the moment and i've
>>>>> been staring at it for way too long now, so i thought: why not put it up
>>>>> here, maybe someone sees what's going on. It's a lenghty story, so if you
>>>>> want to hang on...i'll try to explain :)
>>>>>
>>>>> As you may know, i'm writing a modular synthesizer:
>>>>>
>>>>> http://petervandernoord.nl/patchwork_js (maybe clear your cache if
>>>>> you've been there before)
>>>>>
>>>>> If you click the 'json to patch' button, a testpatch will be set.
>>>>> (Important to know: all custom nodes will be created with the buffer-size
>>>>> that's selected in the pulldown on the right). The patch contains 3 modules
>>>>> (in patchwork, a module can contain one or more audionodes, with the
>>>>> module's in/outputs mapped to certain in/outputs of the containing nodes):
>>>>>
>>>>> - The destination, which contains a normal destinationNode
>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>
>>>>> - a clockmodule. one custom js node
>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>
>>>>> - a triggersequencer, also one custom node.
>>>>> http://localhost/patchworkjs/js/modules/TriggerSequencerModule.js(the audioprocess callback is at the bottom)
>>>>>
>>>>> What is happening in the patch: the clock sends out single values
>>>>> of 1s (all other values are 0) on a given interval (set in BPM). The
>>>>> sequencer checks on every incoming value if that value is >0 AND the
>>>>> previous one was <=0 (i'll call that a clock-pulse). If that is the case,
>>>>> its SequencerParameter will proceed to the next step. A sequencer-parameter
>>>>> (actually it is a LogicSequencerParameter, but that's almost the same - it
>>>>> has one extra method) can be found here:
>>>>>
>>>>> http://localhost/patchworkjs/js/params/SequenceParameter.js
>>>>>
>>>>> It's basically just an array filled ith 0s and 1s (you can set a 1 by
>>>>> clicking somewhere on the sequencer), and increases the current position
>>>>> when it gets a next() command. So, back to the the sequencer module: If it
>>>>> received a clock-pulse, it proceeds the sequencer. Then, if the (new) value
>>>>> of the sequencer-parameter is 1, the sequencer will write a 1 in its
>>>>> outputbuffer as well.
>>>>>
>>>>> My issues:
>>>>> - in the testpatch, both the clockmodule and the seq-module are
>>>>> connected to the output. if you activate some steps in the sequencer, you
>>>>> will hear that the clicks do not run in sync. I have no idea why that is,
>>>>> the stepsequencer writes a 1 in exact the same iteration as it reads the
>>>>> incoming 1s from the clock. In my opinion, they should run exactly in sync.
>>>>> - When you change the buffersize (which is for the customnodes) you
>>>>> will hear that the timedifference between the ticks changes (since there's
>>>>> no clear, you have to refresh the page, set another buffersize and click
>>>>> 'json to patch')
>>>>> - Something else i noticed: when i run just a clock module connected
>>>>> to the output, with a very low buffersize (256, 512), the clock seems to
>>>>> run very, very irregular.
>>>>>
>>>>>
>>>>> So, my main question: Does anyone have any idea why those two modules
>>>>> do not run in sync when both connected to the output?
>>>>>
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>>>
>>>
>>
>
Received on Friday, 3 August 2012 17:12:19 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 3 August 2012 17:12:19 GMT