W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2012

Re: Help needed with a sync-problem

From: Adam Goode <agoode@google.com>
Date: Fri, 3 Aug 2012 13:42:40 -0400
Message-ID: <CAOf41N=mVSqJjEJQFs7=jBYLRBO0g7R2CzibhqVoY=kVP6xj6g@mail.gmail.com>
To: Peter van der Noord <peterdunord@gmail.com>
Cc: Chris Wilson <cwilso@google.com>, Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, public-audio@w3.org
They must add some lag in some cases. You don't have an instantaneous
algorithm running inside there.

Perhaps the issue is that latency is introduced only in one part of a
subgraph? I thought there were some mechanisms for determining the latency
and/or synchronizing such latency across multiple subgraphs?




On Fri, Aug 3, 2012 at 1:40 PM, Peter van der Noord
<peterdunord@gmail.com>wrote:

> I agree fully that it won't be what most developers want or need to do,
> the api will be used for games and site music/effects mostly, but creating
> custom nodes would be my primary focus. To be honest, the list of native
> nodes that i wanted to use has thinned out, due to some behaviours and
> implementations that were not appropriate for what i wanted. That's all
> fine by itself, but if i can't recreate them myself...
>
> I personally don't think "a lot of people won't be using custom nodes
> anyway" is a good argument for not  implement them correctly. If they add
> lag, i can't use them.
>
>
> Peter
>
>
> 2012/8/3 Chris Wilson <cwilso@google.com>
>
>> How would you empower the JS node/DSP API to fix this?
>>
>> I still think, personally, that there's an awful lot of focus on custom
>> processing in our discussions here.  I haven't felt the need to build a
>> JSNode yet - the first one I will build is probably a noise gate/expander,
>> since that's the only thing I can't easily replicate from the nodes already
>> available.  I'm not really convinced that what most application developers
>> want to do - NEED to do - is process audio bits themselves directly.
>>
>>
>> On Fri, Aug 3, 2012 at 10:11 AM, Jussi Kalliokoski <
>> jussi.kalliokoski@gmail.com> wrote:
>>
>>> It's a known and major issue all right, but it's not a bug. There's not
>>> much that can be done about it though, afaict. The processing thread has to
>>> buffer enough data (the buffer size) from the inputs of the JSNode before
>>> its callback can be invoked, and next it just sends an event to the JS
>>> thread to process the buffer. The audio thread, however, can't wait for the
>>> JS thread to process the buffer but instead plays back the previously
>>> processed buffer.
>>>
>>> This is one of the reasons why I think we should focus on empowering the
>>> JS node / DSP API. If you want to add any custom processing to the graph,
>>> you're going to have to adjust the rest of the graph accordingly and you'll
>>> end up with more latency. This means that if you want to do extensive
>>> custom processing you'll probably need to work around the graph or just go
>>> with just the JS and have your own routing which means you're in a much
>>> more flexible place already anyway. I think the graph serves best as an IO
>>> abstraction, and that's the part we should focus on.
>>>
>>> Cheers,
>>> Jussi
>>>
>>>
>>> On Fri, Aug 3, 2012 at 4:16 PM, Peter van der Noord <
>>> peterdunord@gmail.com> wrote:
>>>
>>>> Well, it seems indeed that custom-nodes add a delay-time to the signal.
>>>> I've connected a few bypass modules (they write their input to the output)
>>>> and i'm magically creating an echo...
>>>>
>>>> http://www.petervandernoord.nl/patchwork_js/?patch=2&buffer_size=8192
>>>>
>>>> (to hear sound, you have to select the loaded buffer from the pulldown
>>>> in the buffersource-module)
>>>>
>>>> I'm getting somewhat confused and concerned about this, why does this
>>>> happen and isn't this a major bug/issue?
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>> 2012/8/2 Adam Goode <agoode@google.com>
>>>>
>>>>> I think you can use playbackTime to determine the absolute a-rate time
>>>>> of the beginning of the javascript buffer. But last I checked it wasn't
>>>>> present in webkit.
>>>>>
>>>>> You might be able to count samples, assuming you know the node's
>>>>> noteOn time, to keep track of the a-rate time. But with a short buffer
>>>>> size, sometimes you can have problems as you've noticed.
>>>>>
>>>>>
>>>>> On Thu, Aug 2, 2012 at 4:47 PM, Peter van der Noord <
>>>>> peterdunord@gmail.com> wrote:
>>>>>
>>>>>> Ermmm.....wait, what? And that is intened behavior?
>>>>>>
>>>>>> Peter
>>>>>>
>>>>>>
>>>>>> 2012/8/2 Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
>>>>>>
>>>>>>> Hey Peter!
>>>>>>>
>>>>>>> I think this is because the JSNode has a delay equivalent to the
>>>>>>> buffer size, hence if you have parallel graphs that contain a different
>>>>>>> number of JSNodes, they will arrive to the common destination at a
>>>>>>> different delay.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Jussi
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 2, 2012 at 11:35 PM, Peter van der Noord <
>>>>>>> peterdunord@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm having a strange problem with some signals at the moment and
>>>>>>>> i've been staring at it for way too long now, so i thought: why not put it
>>>>>>>> up here, maybe someone sees what's going on. It's a lenghty story, so if
>>>>>>>> you want to hang on...i'll try to explain :)
>>>>>>>>
>>>>>>>> As you may know, i'm writing a modular synthesizer:
>>>>>>>>
>>>>>>>> http://petervandernoord.nl/patchwork_js (maybe clear your cache if
>>>>>>>> you've been there before)
>>>>>>>>
>>>>>>>> If you click the 'json to patch' button, a testpatch will be set.
>>>>>>>> (Important to know: all custom nodes will be created with the buffer-size
>>>>>>>> that's selected in the pulldown on the right). The patch contains 3 modules
>>>>>>>> (in patchwork, a module can contain one or more audionodes, with the
>>>>>>>> module's in/outputs mapped to certain in/outputs of the containing nodes):
>>>>>>>>
>>>>>>>> - The destination, which contains a normal destinationNode
>>>>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>>>>
>>>>>>>> - a clockmodule. one custom js node
>>>>>>>> http://localhost/patchworkjs/js/modules/DestinationModule.js
>>>>>>>>
>>>>>>>> - a triggersequencer, also one custom node.
>>>>>>>> http://localhost/patchworkjs/js/modules/TriggerSequencerModule.js(the audioprocess callback is at the bottom)
>>>>>>>>
>>>>>>>> What is happening in the patch: the clock sends out single values
>>>>>>>> of 1s (all other values are 0) on a given interval (set in BPM). The
>>>>>>>> sequencer checks on every incoming value if that value is >0 AND the
>>>>>>>> previous one was <=0 (i'll call that a clock-pulse). If that is the case,
>>>>>>>> its SequencerParameter will proceed to the next step. A sequencer-parameter
>>>>>>>> (actually it is a LogicSequencerParameter, but that's almost the same - it
>>>>>>>> has one extra method) can be found here:
>>>>>>>>
>>>>>>>> http://localhost/patchworkjs/js/params/SequenceParameter.js
>>>>>>>>
>>>>>>>> It's basically just an array filled ith 0s and 1s (you can set a 1
>>>>>>>> by clicking somewhere on the sequencer), and increases the current position
>>>>>>>> when it gets a next() command. So, back to the the sequencer module: If it
>>>>>>>> received a clock-pulse, it proceeds the sequencer. Then, if the (new) value
>>>>>>>> of the sequencer-parameter is 1, the sequencer will write a 1 in its
>>>>>>>> outputbuffer as well.
>>>>>>>>
>>>>>>>> My issues:
>>>>>>>> - in the testpatch, both the clockmodule and the seq-module are
>>>>>>>> connected to the output. if you activate some steps in the sequencer, you
>>>>>>>> will hear that the clicks do not run in sync. I have no idea why that is,
>>>>>>>> the stepsequencer writes a 1 in exact the same iteration as it reads the
>>>>>>>> incoming 1s from the clock. In my opinion, they should run exactly in sync.
>>>>>>>> - When you change the buffersize (which is for the customnodes) you
>>>>>>>> will hear that the timedifference between the ticks changes (since there's
>>>>>>>> no clear, you have to refresh the page, set another buffersize and click
>>>>>>>> 'json to patch')
>>>>>>>> - Something else i noticed: when i run just a clock module
>>>>>>>> connected to the output, with a very low buffersize (256, 512), the clock
>>>>>>>> seems to run very, very irregular.
>>>>>>>>
>>>>>>>>
>>>>>>>> So, my main question: Does anyone have any idea why those two
>>>>>>>> modules do not run in sync when both connected to the output?
>>>>>>>>
>>>>>>>>
>>>>>>>> Peter
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Received on Friday, 3 August 2012 17:43:08 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 3 August 2012 17:43:09 GMT