W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2012

Re: MIDI Tracks and Sequences

From: James Ingram <j.ingram@netcologne.de>
Date: Sun, 16 Sep 2012 19:06:14 +0200
Message-ID: <50560706.6070400@netcologne.de>
To: Chris Wilson <cwilso@google.com>
CC: public-audio@w3.org, Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, Joseph Berkovitz <joe@noteflight.com>
Hi Chris, Jussi, all

I have now understood "the critical point", that sendMIDIMessage is 
supposed to schedule the message rather than send it immediately.
I think its unlikely that MidiBridge is implementing this, but have 
written privately to abudaan to find out. Even if he has implemented 
timestamps in sendMIDIMessage, we also need to be sure he's using the 
DOMHRT timer.

https://gist.github.com/3728539:
On my machine, the schedule deviation gradually increases (irregularly), 
and I noticed that it was getting increasingly erratic after about 
160000 ms (2min 40sec), so decided to let it run longer to see what 
happened. I've now doubled the length of the sequence (see test 7 [1]). 
If you don't hit the stop button, the test will now run just short of 
350000ms (5min 50sec).
After about 240000ms a pattern emerges. I don't know what the pattern 
means, but have added a "devDelta" field to the log showing the 
difference between successive deviations, so that you can see it more 
clearly. The "deviation" field is the "sequence deviation" field, as before.
The sequence itself contains a repeating 4-duration cycle of 50, 50, 50, 
200ms, and by the end of the performance the scheduling deviations have 
also settled down into a 4-value cycle: 350, 300, 250, 200 or 351, 301, 
251, 201 or similar. The 50ms differences seem quite constant, but the 
deviations are still gradually increasing as a whole.

And there's another thing that I still don't quite get.
If the system is supposed to send the message at its timestamp time, how 
does it know when the performance started? I suspect that there is an 
error in test 7 (left over from my previous tests) , and that the 
timestamp sent to the system should actually be simple DOMHRT time 
(relative to session start), rather than performance-relative time. Is 
that right?
This would not make any difference to the other code in test 7, but it 
sure would to the time at which the system is supposed to send the message.
If I'm right, then all the timestamps in test 7 would be in the past as 
far as the system is concerned. So the system would be sending the 
messages immediately (which is what we are expecting anyway with 
PREQUEUE=0). This error (if it is one) is only going to show up when the 
value of PREQUEUE is increased, and we start trying to schedule messages 
in the future.

> Things like this is why we need native implementations of the MIDI API.
Any chance of that happening in the near future?

best,
James

[1] http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence7.html


Am 15.09.2012 17:49, schrieb Chris Wilson:
> On Sat, Sep 15, 2012 at 5:12 AM, James Ingram <j.ingram@netcologne.de 
> <mailto:j.ingram@netcologne.de>> wrote:
>
>         It really, really doesn't need to be recursive.
>
>     I don't think you can remove the recursion, and stay accurate at
>     the same time. See below.
>
>
> You can, in a conformant implementation.  That's the whole point. 
>  Recursion in this way would be very bad, for precisely the reasons 
> you're running into.
>
>     You are right, of course, using setInterval was an artefact. I
>     thought it might help keep the stack size down if I could
>     explicitly clear something, but using setTimeout actually makes no
>     difference (I just tried it).
>
>
> Stack scope and timers here are unrelated; setTimeout vs. setInterval 
> is not relevant to your scope issue, your recursion is.  setInterval 
> vs setTimeout is just a programming convenience.
>
>         Additionally, you are only sending one message per tick;
>         that's not really the point.  You should be sending any
>         messages that are between this interval call and the next, to
>         get them queued up;
>         ... more like this: https://gist.github.com/3722988.
>
>
>     That's more or less what Joe suggested too.
>     But if you send all messages within the INTERVAL as fast as
>     possible (in the while{} loop) then you lose accuracy. 
>
>
> No, you don't.  This is a misunderstanding of what sendMIDIMessage is 
> supposed to do - it SCHEDULES the message, at the timestamp in the 
> event.  It is not supposed to send the message immediately, if the 
> timestamp is in the future.  (Caveat: I have no idea if MIDIBridge 
> properly implements timestamps or not - this may simply not work 
> today.  Things like this is why we need native implementations of the 
> MIDI API.  In fact, based on behavior in your example, I would say 
> MIDIBridge does NOT support timestamps.  Try 
> https://gist.github.com/3728539 - this should make up for it as best 
> it can.)  This is the critical point to understand.
>
>     As far as I can see, the delay between messages inside the
>     INTERVAL is being ignored, and everything gets scheduled on a grid
>     of INTERVAL milliseconds (plus or minus a few nano-seconds). In
>     your code that's every 200ms. That's also what I hear in test 6
>     [1], where I have simply replaced my sendMIDISequence() function
>     by your code 
>
>
> Everything gets SCHEDULED on an interval of 200 ms or so; however, in 
> a conformant implementation, the note would be SENT at the appropriate 
> time.  I should not have left your logging code in my example, because 
> it's misleading; your logging code that lists a "deviation" is just 
> capturing when the note scheduling is taking place, not when the note 
> should actually be sent.  You don't have access to that information 
> (because it's below the API level); the best you could do is loop an 
> actual MIDI cable back from an OUT port to an IN port and watch the 
> timestamps on the inputs (and that will have some of its own latency).
>
>     (remember, my recursive function was working with an accuracy of 1
>     millisecond). 
>
>
> Because you were choking the setTimeout queue.  setTimeout(0) WORKS - 
> but it's not a good programming form, as it's burning processing power.
>
>     [Actually, that's not quite true: I have seen rare glitches in
>     test 5 (maybe once in 3 minutes) in which 2-3 notes deviate by
>     more than 1 millisecond. Obviously, setInterval sometimes takes a
>     rest...]
>
>
> Sure.  You might have relayouts happening, or other things the system 
> has to take into account.  That's precisely why the MIDI API has its 
> own timing system; setInterval/Timeout is unreliable.
>
>     Interestingly, the scheduling in test 6 (your code) also decays
>     over time, and the individual note-clusters spread out a bit. 
>
>
> I'm not sure what you mean by "decays over time".  All the "deviation" 
> values should be listed between 200 and 400, and they are.
>
> I do note that the initial ones are off, because you need to call your 
> (anonymous) scheduling function exactly when the DOMHRT sequencer 
> clock starts (because you're missing an opportunity to schedule the 
> first 200ms of events).
>
>     Looking at the log, there's also some drift in where setInterval
>     is starting each INTERVAL. That's obviously what you meant by
>
>         and really, you need some lookahead past that, in case the
>         timers are skewed a bit (aka the setInterval callback is late).
>
>
> Yes, that's expected.
>
>     I'm not at all happy with an indeterminacy of around 200ms, and I
>     know that passing small delays to setInterval  is something one
>     shouldn't expect to work. 
>
>
> That is correct.  But you have the underlying scheduling of the 
> sendMIDIMessage implementation to take care of the indeterminacy below 
> 200ms; in fact, you could schedule the entire sequence at one go.  But 
> then you couldn't change tempo or cancel the output.
>
>     So there's no way I can get to anything like 1 millisecond
>     accuracy by using setInterval or setTimeout non-recursively.
>
>
> Recursion is not relevant here - you can't get true 1ms accuracy using 
> setInterval or setTimeout in any case.  This is why we have timestamps 
> built in to the API, and that level of accuracy is left to the 
> implementation of the MIDI API.
>


-- 
www.james-ingram-act-two.de
Received on Sunday, 16 September 2012 17:06:57 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Sunday, 16 September 2012 17:06:57 GMT