Re: Help needed with a sync-problem

On Fri, Aug 3, 2012 at 10:40 AM, Peter van der Noord
<peterdunord@gmail.com>wrote:

> I agree fully that it won't be what most developers want or need to do,
> the api will be used for games and site music/effects mostly, but creating
> custom nodes would be my primary focus. To be honest, the list of native
> nodes that i wanted to use has thinned out, due to some behaviours and
> implementations that were not appropriate for what i wanted. That's all
> fine by itself, but if i can't recreate them myself...
>
> I personally don't think "a lot of people won't be using custom nodes
> anyway" is a good argument for not  implement them correctly. If they add
> lag, i can't use them.
>

Peter, as Jussi has pointed out.  This is a sad fact of life and "not a
bug".  If you read more carefully what Jussi says, this lag/latency is not
something we can somehow fix by creating a different API.  This is the way
that a real-time audio thread interacting with a main thread (which can
have its own delays such as garbage collection) must be.  It's a law of
physics.

Chris


>
>
> Peter
>
>
> 2012/8/3 Chris Wilson <cwilso@google.com>
>
>> How would you empower the JS node/DSP API to fix this?
>>
>> I still think, personally, that there's an awful lot of focus on custom
>> processing in our discussions here.  I haven't felt the need to build a
>> JSNode yet - the first one I will build is probably a noise gate/expander,
>> since that's the only thing I can't easily replicate from the nodes already
>> available.  I'm not really convinced that what most application developers
>> want to do - NEED to do - is process audio bits themselves directly.
>>
>>
>> On Fri, Aug 3, 2012 at 10:11 AM, Jussi Kalliokoski <
>> jussi.kalliokoski@gmail.com> wrote:
>>
>>> It's a known and major issue all right, but it's not a bug. There's not
>>> much that can be done about it though, afaict. The processing thread has to
>>> buffer enough data (the buffer size) from the inputs of the JSNode before
>>> its callback can be invoked, and next it just sends an event to the JS
>>> thread to process the buffer. The audio thread, however, can't wait for the
>>> JS thread to process the buffer but instead plays back the previously
>>> processed buffer.
>>>
>>> This is one of the reasons why I think we should focus on empowering the
>>> JS node / DSP API. If you want to add any custom processing to the graph,
>>> you're going to have to adjust the rest of the graph accordingly and you'll
>>> end up with more latency. This means that if you want to do extensive
>>> custom processing you'll probably need to work around the graph or just go
>>> with just the JS and have your own routing which means you're in a much
>>> more flexible place already anyway. I think the graph serves best as an IO
>>> abstraction, and that's the part we should focus on.
>>>
>>> Cheers,
>>> Jussi
>>>
>>>
>>> On Fri, Aug 3, 2012 at 4:16 PM, Peter van der Noord <
>>> peterdunord@gmail.com> wrote:
>>>
>>>> Well, it seems indeed that custom-nodes add a delay-time to the signal.
>>>> I've connected a few bypass modules (they write their input to the output)
>>>> and i'm magically creating an echo...
>>>>
>>>> http://www.petervandernoord.nl/patchwork_js/?patch=2&buffer_size=8192
>>>>
>>>> (to hear sound, you have to select the loaded buffer from the pulldown
>>>> in the buffersource-module)
>>>>
>>>> I'm getting somewhat confused and concerned about this, why does this
>>>> happen and isn't this a major bug/issue?
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>> 2012/8/2 Adam Goode <agoode@google.com>
>>>>
>>>>> I think you can use playbackTime to determine the absolute a-rate time
>>>>> of the beginning of the javascript buffer. But last I checked it wasn't
>>>>> present in webkit.
>>>>>
>>>>> You might be able to count samples, assuming you know the node's
>>>>> noteOn time, to keep track of the a-rate time. But with a short buffer
>>>>> size, sometimes you can have problems as you've noticed.
>>>>>
>>>>>
>>>>> On Thu, Aug 2, 2012 at 4:47 PM, Peter van der Noord <
>>>>> peterdunord@gmail.com> wrote:
>>>>>
>>>>>> Ermmm.....wait, what? And that is intened behavior?
>>>>>>
>>>>>> Peter
>>>>>>
>>>>>>
>>>>>> 2012/8/2 Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
>>>>>>
>>>>>>> Hey Peter!
>>>>>>>
>>>>>>> I think this is because the JSNode has a delay equivalent to the
>>>>>>> buffer size, hence if you have parallel graphs that contain a different
>>>>>>> number of JSNodes, they will arrive to the common destination at a
>>>>>>> different delay.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Jussi
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 2, 2012 at 11:35 PM, Peter van der Noord <
>>>>>>> peterdunord@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm having a strange problem with some signals at the moment and
>>>>>>>> i've been staring at it for way too long now, so i thought: why not put it
>>>>>>>> up here, maybe someone sees what's going on. It's a lenghty story, so if
>>>>>>>> you want to hang on...i'll try to explain :)
>>>>>>>>
>>>>>>>> As you may know, i'm writing a modular synthesizer:
>>>>>>>>
>>>>>>>> http://petervandernoord.nl/patchwork_js (maybe clear your cache if
>>>>>>>> you've been there before)
>>>>>>>>
>>>>>>>> If you click the 'json to patch' button, a testpatch will be set.
>>>>>>>> (Important to know: all custom nodes will be created with the buffer-size
>>>>>>>> that's selected in the pulldown on the right). The patch contains 3 modules
>>>>>>>> (in patchwork, a module can contain one or more audionodes, with the
>>>>>>>> module's in/outputs mapped to certain in/outputs of the containing nodes):
>>>>>>>>
>>>>>>>> - The destination, which contains a normal destinationNode
>>>>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>>>>
>>>>>>>> - a clockmodule. one custom js node
>>>>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>>>>
>>>>>>>> - a triggersequencer, also one custom node.
>>>>>>>> http://localhost/patchworkjs/js/modules/TriggerSequencerModule.js(the audioprocess callback is at the bottom)
>>>>>>>>
>>>>>>>> What is happening in the patch: the clock sends out single values
>>>>>>>> of 1s (all other values are 0) on a given interval (set in BPM). The
>>>>>>>> sequencer checks on every incoming value if that value is >0 AND the
>>>>>>>> previous one was <=0 (i'll call that a clock-pulse). If that is the case,
>>>>>>>> its SequencerParameter will proceed to the next step. A sequencer-parameter
>>>>>>>> (actually it is a LogicSequencerParameter, but that's almost the same - it
>>>>>>>> has one extra method) can be found here:
>>>>>>>>
>>>>>>>> http://localhost/patchworkjs/js/params/SequenceParameter.js
>>>>>>>>
>>>>>>>> It's basically just an array filled ith 0s and 1s (you can set a 1
>>>>>>>> by clicking somewhere on the sequencer), and increases the current position
>>>>>>>> when it gets a next() command. So, back to the the sequencer module: If it
>>>>>>>> received a clock-pulse, it proceeds the sequencer. Then, if the (new) value
>>>>>>>> of the sequencer-parameter is 1, the sequencer will write a 1 in its
>>>>>>>> outputbuffer as well.
>>>>>>>>
>>>>>>>> My issues:
>>>>>>>> - in the testpatch, both the clockmodule and the seq-module are
>>>>>>>> connected to the output. if you activate some steps in the sequencer, you
>>>>>>>> will hear that the clicks do not run in sync. I have no idea why that is,
>>>>>>>> the stepsequencer writes a 1 in exact the same iteration as it reads the
>>>>>>>> incoming 1s from the clock. In my opinion, they should run exactly in sync.
>>>>>>>> - When you change the buffersize (which is for the customnodes) you
>>>>>>>> will hear that the timedifference between the ticks changes (since there's
>>>>>>>> no clear, you have to refresh the page, set another buffersize and click
>>>>>>>> 'json to patch')
>>>>>>>> - Something else i noticed: when i run just a clock module
>>>>>>>> connected to the output, with a very low buffersize (256, 512), the clock
>>>>>>>> seems to run very, very irregular.
>>>>>>>>
>>>>>>>>
>>>>>>>> So, my main question: Does anyone have any idea why those two
>>>>>>>> modules do not run in sync when both connected to the output?
>>>>>>>>
>>>>>>>>
>>>>>>>> Peter
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>