- From: James Ingram <j.ingram@netcologne.de>
- Date: Tue, 05 Jun 2012 11:14:58 +0200
- To: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
- CC: Chris Wilson <cwilso@google.com>, public-audio@w3.org
Hi Jussi, Chris, > Jussi: Garbage collection isn't necessarily a problem, since > implementations will probably just use JS wrappers for the messages > and the data will actually be stored in an underlying struct, and MIDI > isn't exactly one of the highest traffic protocols anyway. I was thinking of situations in which there have to be large numbers of messages in memory waiting to be sent (maybe tens or even hundreds of thousands of them). But there are probably strategies for minimizing the problem (see below). > James: I've been investigating the limitations on timing in > Javascript applications: > 1. According to the W3C standard, the delay set in setTimeout() > will never be less than 4 milliseconds [2]. In practice, the lower > limit can be much larger [3]. > 2. The time interval in setInterval() is measured in milliseconds, > so there are limits on the rates at which it can be used to send > MIDI Clock events. (MIDI defines the rate at which MIDI Clocks are > sent in *microseconds*. MIDI-Clocks are usually sent event every > 20 milliseconds or so.) > > > Jussi: Yes, this is exactly why timestamps are quite essential in a > MIDI API that will be in a JavaScript environment, they allow you to > make the messages happen as accurately as possible, i.e. ahead of > time, so that you don't have to rely on unreliable timing systems such > as setTimeout(). It wouldn't be the end of the world if we had to rely on setTimeout() and setInterval(). There are going to be limitations somewhere. Its just that we have to know where they are. (One writes differently for a xylophone than for a tuba.) But of course we want the highest accuracy we can get out of these machines. > Chris: Actually, I was wondering if it would be possible to create > arrays of MIDIMessages, to buffer them up with timestamps. (see below) > Jussi: I'd go for letting the cows pave the path again... :-) and > When we have actual data on what kind of usage patterns arise we can > start thinking about things like this, imho. Okay you asked for it. :-) I'm programming a MIDI-player. This reads MIDI data from an SVG-MIDI score displayed in the browser (like the one at [1]), and sends it to a MIDI output device. The player has controls like a Flash player (stop, go, pause, set start position, set end position, goto start, goto end). There will also be channel filters to allow the user to select which channels (=voices) to play, and some kind of indicator in the score showing the current performance position while playing. The test score uses three MIDI channels, one per staff. I'm still learning Javascript so, as an exercise which I knew was going to have to be re-written, I first tried the naive approach to see what would happen (I wanted to hear something): 1. read the MIDI data from the DOM 2. convert it to midiMessages 3. put all the messages in a single list of midiEvents with a _delay_ in milliseconds between them. 4. send the list to midiOut using setTimeout(). (a midiEvent is a list of midiMessages that are to be sent "at the same time". But note that they are actually sent in a known order, so patch change messages in the midiEvent should affect messages in the same channel lower down the midiEvent.) Result: a) It takes about 35 seconds for my machine (which is quite fast) to read the DOM and create the list of midiEvents. That can probably be improved as my Javascript improves, but reading the DOM is always going to take significant amounts of time. b) The performance was considerably slower than the reference mp3 (which is the conversion of a MIDI file I made using C#). Obviously this is because the midiEvent list contained lots of delays less than 4ms. I'm currently working on the user interface, but will soon be tackling the MIDI again. My current plan is as follows: 1. Do the DOM reading and performance in a separate threads. (I haven't used, or even properly looked at Javascript Workers yet, so I don't know if this is going to work.) I imagine reading ca 5 seconds from the DOM, giving the messages to the performing thread, and going back to reading the DOM. When the performing thread is ready, the DOM thread gives it what its got and then goes back to reading. Note that reading the DOM and creating midiMessages are two separate operations. It might be better to let the performance thread do the midiMessage construction, so that the DOM thread has less to do. That would mean passing the midiMessage _parameters_ to the performing thread rather than the midiMessages themselves. 2. Chord symbols contain MIDI info which is fundamentally in two parallel streams: a) a stream containing ChordOns, ChordOffs and patch changes. (Chord symbols can represent ornaments.) b) a stream containing slider (controller) info. I want to try writing a MIDIChord.play(midiOut) function which would actually use two threads to send the two streams. This makes _delays_ much more controllable. There would not be an undue proliferation of Workers, because one only really needs two per channel. Its quite sufficient to send sliderEvents every 50ms or so (20 per second). Each sliderEvent can contain messages for all the sliders currently in use in the chord. I still don't see why I need to bring absolute time into this. Delays are something else. I'm quite happily using millisecond units, but you might have a good reason for wanting to use microseconds. all the best, James [1] http://james-ingram-act-two.de/compositions/study2/study2b/Study2b3.html (For some reason, as far as I know, the font is only loading properly in Chrome and Safari at the moment... I'm using Chrome, and not bothering too much about the other browsers yet.)
Received on Tuesday, 5 June 2012 09:18:03 UTC