Re: Timing limitations when programming MIDI with Javascript

Hi Jussi, Chris,

> Jussi: Garbage collection isn't necessarily a problem, since 
> implementations will probably just use JS wrappers for the messages 
> and the data will actually be stored in an underlying struct, and MIDI 
> isn't exactly one of the highest traffic protocols anyway.
I was thinking of situations in which there have to be large numbers of 
messages in memory waiting to be sent (maybe tens or even hundreds of 
thousands of them). But there are probably strategies for minimizing the 
problem (see below).

>     James: I've been investigating the limitations on timing in
>     Javascript applications:
>     1. According to the W3C standard, the delay set in setTimeout()
>     will never be less than 4 milliseconds [2]. In practice, the lower
>     limit can be much larger [3].
>     2. The time interval in setInterval() is measured in milliseconds,
>     so there are limits on the rates at which it can be used to send
>     MIDI Clock events. (MIDI defines the rate at which MIDI Clocks are
>     sent in *microseconds*. MIDI-Clocks are usually sent event every
>     20 milliseconds or so.)
>
>
> Jussi: Yes, this is exactly why timestamps are quite essential in a 
> MIDI API that will be in a JavaScript environment, they allow you to 
> make the messages happen as accurately as possible, i.e. ahead of 
> time, so that you don't have to rely on unreliable timing systems such 
> as setTimeout().
It wouldn't be the end of the world if we had to rely on setTimeout() 
and setInterval(). There are going to be limitations somewhere. Its just 
that we have to know where they are. (One writes differently for a 
xylophone than for a tuba.) But of course we want the highest accuracy 
we can get out of these machines.

> Chris: Actually, I was wondering if it would be possible to create 
> arrays of MIDIMessages, to buffer them up with timestamps.
(see below)

> Jussi: I'd go for letting the cows pave the path again... 
:-) and
> When we have actual data on what kind of usage patterns arise we can 
> start thinking about things like this, imho.

Okay you asked for it. :-)

I'm programming a MIDI-player.
This reads MIDI data from an SVG-MIDI score displayed in the browser 
(like the one at [1]), and sends it to a MIDI output device.
The player has controls like a Flash player (stop, go, pause, set start 
position, set end position, goto start, goto end). There will also be 
channel filters to allow the user to select which channels (=voices) to 
play, and some kind of indicator in the score showing the current 
performance position while playing.
The test score uses three MIDI channels, one per staff.

I'm still learning Javascript so, as an exercise which I knew was going 
to have to be re-written, I first tried the naive approach to see what 
would happen (I wanted to hear something):
1. read the MIDI data from the DOM
2. convert it to midiMessages
3. put all the messages in a single list of midiEvents with a  _delay_ 
in milliseconds between them.
4. send the list to midiOut using setTimeout().

(a midiEvent is a list of midiMessages that are to be sent "at the same 
time". But note that they are actually sent in a known order, so patch 
change messages in the midiEvent should affect messages in the same 
channel lower down the midiEvent.)

Result:
a) It takes about 35 seconds for my machine (which is quite fast) to 
read the DOM and create the list of midiEvents. That can probably be 
improved as my Javascript improves, but reading the DOM is always going 
to take significant amounts of time.
b) The performance was considerably slower than the reference mp3 (which 
is the conversion of a MIDI file I made using C#). Obviously this is 
because the midiEvent list contained lots of delays less than 4ms.

I'm currently working on the user interface, but will soon be tackling 
the MIDI again.

My current plan is as follows:
1. Do the DOM reading and performance in a separate threads. (I haven't 
used, or even properly looked at Javascript Workers yet, so I don't know 
if this is going to work.) I imagine reading ca 5 seconds from the DOM, 
giving the messages to the performing thread, and going back to reading 
the DOM. When the performing thread is ready, the DOM thread gives it 
what its got and then goes back to reading.
Note that reading the DOM and creating midiMessages are two separate 
operations. It might be better to let the performance thread do the 
midiMessage construction, so that the DOM thread has less to do. That 
would mean passing the midiMessage _parameters_ to the performing thread 
rather than the midiMessages themselves.

2. Chord symbols contain MIDI info which is fundamentally in two 
parallel streams:
a) a stream containing ChordOns, ChordOffs and patch changes. (Chord 
symbols can represent ornaments.)
b) a stream containing slider (controller) info.
I want to try writing a MIDIChord.play(midiOut) function which would 
actually use two threads to send the two streams. This makes _delays_ 
much more controllable. There would not be an undue proliferation of 
Workers, because one only really needs two per channel.
Its quite sufficient to send sliderEvents every 50ms or so (20 per 
second). Each sliderEvent can contain messages for all the sliders 
currently in use in the chord.

I still don't see why I need to bring absolute time into this.  Delays 
are something else.  I'm quite happily using millisecond units, but you 
might have a good reason for wanting to use microseconds.

all the best,
James

[1] http://james-ingram-act-two.de/compositions/study2/study2b/Study2b3.html
(For some reason, as far as I know, the font is only loading properly in 
Chrome and Safari at the moment... I'm using Chrome, and not bothering 
too much about the other browsers yet.)

Received on Tuesday, 5 June 2012 09:18:03 UTC