- From: James Ingram <j.ingram@netcologne.de>
- Date: Wed, 12 Sep 2012 23:58:16 +0200
- To: Chris Wilson <cwilso@google.com>
- CC: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, public-audio@w3.org
Hi Chris, Sorry, its me that's confused! :-) > My suggested implementation tactic would be to use setInterval to > pre-schedule MIDI events in the send message queue, keeping some > arbitrary interval of time (a tenth of a second or so?) ahead of the > current DOMHRT time; this slower loop can then be cancelled whenever > you hit stop. I'll try that. Thanks very much for the suggestion. By the way, DOMHRT time is AudioContext.currentTime, right? best, James Am 12.09.2012 19:47, schrieb Chris Wilson: > James, > I'm quite confused. sendTimestampedMIDIMessage in that shim is not a > function you should have to write or edit; the Web MIDI API has a > "send this message, at this absolute time" capability. You shouldn't > need to deal with clock drift; in a real, native implementation of the > Web MIDI API, a lower-level timing system would be handling this, not > Javascript's setTimeout/setInterval; much like the Web Audio API has > its own system for scheduling. This is going to be a weak point in > any polyfill (the one you point to, and the one I'm writing) - you > simply cannot get access to a high-enough resolution clock to properly > implement the timestamps, and none of the current ways to access MIDI > in Javascript that I know of expose a higher-res timing system. > > For playing/pausing/stopping, the only thing I think we may be missing > is the ability to cancel all scheduled MIDI messages, although before > imposing that I need to look more closely at how the underlying > systems support such a beast; you may need to close the device in > order to clear it. > > My suggested implementation tactic would be to use setInterval to > pre-schedule MIDI events in the send message queue, keeping some > arbitrary interval of time (a tenth of a second or so?) ahead of the > current DOMHRT time; this slower loop can then be cancelled whenever > you hit stop. > > On Wed, Sep 12, 2012 at 10:10 AM, James Ingram <j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>> wrote: > > > Hi Chris, Jussi, all, > > As you pointed out, this kind of thing may well belong in a > standard library, but I think it might be worth taking a closer > look at the problems that are going to have to be solved, in case > we discover that lower level solutions might be better after all. > > As I said, I'm currently writing a sequencer in Javascript, using > JazzMIDIBridge [5] to emulate the future Web MIDI API. > > I've created my own definition of a Sequence, and can now load > such Sequences from files. The immediate problem is now to write a > simple > sequence.play() > function which will play the sequence, from beginning to end, as > accurately as possible. > > In general, I expect sequences to be large (there should be no > limit, in principle, to their duration), and to contain lots of > continuous control data. So I'm creeping up on the problem, by > writing a few test files. (I'm working on Windows, and using the > latest Chrome browser.) > > Test 1 [1] : Starting with the JazzMIDIBridge example5 file (see [5]): > The original example5 file uses setTimeout() naively to simulate a > sequencer, and the first thing I wanted to do was implement the > approach described in [6] to cancel out the delays caused by other > processes running in the system. In other words, the delay passed > to setTimeout() is adjusted in this test so that > (systemTime + delay) is equal to the MIDIMessage.timestamp. > Having done that, I set the code up to play a simple sequence of > 50 NOTE_ON messages at 200 milliseconds per message. > The result is much more accurate than it would have been without > adjusting the delay. > I'm not too worried about Date().getTime()'s (lack of) accuracy. > Presumably the Web Audio timer would be better, but the principle > would not change. My machine has no problem playing this test > sequence completely accurately (according to the system clock). > > Test 2 [2] : the same code as test 1, except that it tries to play > 1000 NOTE_ON messages at 100 milliseconds per message. That should > take 100 seconds. As can be seen from the output, the delay value > being passed to setTimeout() tends to get smaller over time, > because the amount of time being used for other processes is > accumulating. Eventually, the delay is going to go negative, and > it will no longer be possible to honour the message's timestamp at > all. That can be seen happening, on my machine, in Test 3. > > Test 3 [3]: is the same code as tests 1 & 2, but, with 2000 > NOTE_ONs at 50 ms per message. > (I'm aware that there are issues connected with passing small > delay values to setTimeout().) > > Test 4 [4]: This test uses 500 repeats of the sequence 50, 50, 50, > 200 milliseconds. > I'm trying to play the sequence "as accurately as possible", and > tests 1-3 show that no computer on earth is going to be able to > play long, dense sequences accurately according to the timestamps > forever if it is simultaneously doing other things. (Is that right?) > So I've started using "time drift" or "local time": The base time > from which the timestamps are measured is allowed to drift, and > the drift occurs where it is least noticed -- during the longer > notes. Adding 10ms to a long duration is less noticeable than > adding it to a short one. I don't really care if a performance > which should last several minutes, is actually a few seconds longer. > So the function which uses the timestamp to calculate the delay > has to use the previous timestamp as well, in order to know > whether it can add a bit of drift or not. The previous timestamp > is therefore cached. > The algorithm for adding drift could probably be improved, but the > fact of having to add drift, and the algorithm's dependency on the > durations between timestamps is not going to change. > As defined, this sequence should should take exactly 175000 > miliseconds (just short of 3 minutes), but the last time I ran it > it took about 4 seconds longer than that. But at least it sounded > okay (if you like that sort of thing) and got that far without > wanting to go backwards in time! :-) > > Am I missing something? I'm not at all sure that this is > fundamentally the right way to go. I was rather hoping that there > could be accurate API functions for > playing/pausing/stopping/rewinding/ MIDI sequences. But that would > mean agreeing on Track and Sequence definitions. Which is where I > came in. :-) > > all the best, > James > > [1] > http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence1.html <http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > [2] > http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence2.html <http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > [3] > http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence3.html <http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > [4] > http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html > > [5] https://github.com/abudaan/JazzMIDIBridge > [6] http://www.sitepoint.com/creating-accurate-timers-in-javascript/ > > > Am 12.09.2012 01:14, schrieb Chris Wilson: > > Hey James- > > Actually, sendMIDIMessage takes MIDIMessages, which can have a > timestamp - it's not an "absolute" timestamp in the sense of > being relative to the UNIX epoch or anything, but you can in > fact send an entire buffer of MIDIMessages with their > timestamps to be sent at the same time. > > This doesn't address wanting to change tempo dynamically (or > pausing), of course; but you absolutely can do this. I'm not > sure it's what you'd want in most cases (other than simple > playback), but it is definitely there. > > On Tue, Sep 11, 2012 at 2:00 AM, James Ingram > <j.ingram@netcologne.de <mailto:j.ingram@netcologne.de> > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>>> wrote: > > Hi Chris, Jussi, all, > > Thanks, Chris, for this answer. I could certainly live > with using > a dedicated Javascript library which provided Tracks and > Sequences. Probably, some libraries will be more suitable > for some > tasks than others, and its best to leave that flexibility > outside > the MIDI API. > And I take your point that the current API needs to get locked > down before we start talking about v2 (libraries). > > But I'm still wondering if the API is really complete as it > stands. I'm currently wrestling with the problem of accurate > timing, and am asking myself why this is such a hassle. If > there > was an accurate > MIDIOutput.sendAtAbsoluteTime(midiIMessage, absoluteTime) > then the underlying code could take advantage of existing > timing > facilities, and the API's user (me) could get on with solving > other problems. I suspect that timing problems in > particular would > be best solved at a low level... > > best, > James > > Am 11.09.2012 00:48, schrieb Chris Wilson: > > Hey James - > > A while ago, I poked at implementing Standard MIDI File > reading in Javascript > (https://github.com/cwilso/Standard-MIDI-File-reader), > and my > conclusion was basically that it would be best > provided as a > library and/or as baked into the standard <audio> player > engine in the browser, rather than centering the MIDI > API on > it. There are a lot of challenges with doing > rewinding, for > example, and seeking. There is also a lot of dynamic > control > hooking in to tempo changes, live information exposure of > events, etc., that any "serious" sequencer would > probably want > to have. Even in that relatively naive reader, there > are a > bunch of different types of data - it has more > exposure than > the MIDI API does by itself. Once we've got the MIDI API > locked down and implemented, I'll probably go back and > flesh > out my SMF reader as a library for reading and playing > SMFs, > and we can talk about if it's a useful v2 standard > library. > > -C > > On Mon, Sep 10, 2012 at 5:06 AM, James Ingram > <j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de> <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>> > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de> > > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>>>> wrote: > > Hi Jussi, > > I think support for Sequences would be useful (and > easy to > provide), even if we are not talking about file > output. > Loading sequences from files is very common in MIDI > programs, and > I see no reason why that should not be done on the > web. So it > would be nice to have a standard Sequence > definition with > which to > work. I'm not saying that Sequences necessarily > have to be > loaded > from SMFs, but if the Sequence definition existed, > it could > obviously be given a loadFromSMF(url) member function. > > Also, if we had a Sequence object, it would wrap > the output > device, and play(), stop(), rewind() etc. > functions could be > provided quite cheaply out of the box. That might > make Web > MIDI > even more attractive. :-) > > But maybe I'm too close to the problem. Is there > anyone > else out > there who needs to send a Sequence to the output > device? > > best, > James > > > Hi James, > > A side note: the current spec has the notion of > MIDIEvent as a > DOM event that occurs when MIDIMessage(s) are > received. > > The toolkit you're referring to deals both > with MIDI > files (SMF) > and MIDI devices. Our API is designed to work with > just devices, > and MIDI devices are just I/O, there's no > notion of > high level > concepts such as tracks (although there's > channels, > but that's > already in the spec) or sequences, unlike SMF > does. It > would seem > a bit weird to me to complicate the spec with such > high-level > concepts as we don't even support reading / > writing > SMF files, > nor do I think we should as the intent of the > device is > explicitly to allow communication with MIDI > devices. > > Cheers, > Jussi > > On Mon, Sep 10, 2012 at 12:32 PM, James Ingram > <j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de> > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>> > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de> > > <mailto:j.ingram@netcologne.de > <mailto:j.ingram@netcologne.de>>>> wrote: > > Hi Chris, all, > > Chris Wilson said (apropos Bug 18764) > > I'd like to explicitly ask for MORE > feedback > from developers > using MIDI today on this issue. > > I'm following the Bug 18764 debate very > closely, > but don't > have the in-depth knowledge of MIDI's > finer points to > contribute much to the subject. All I > want as a > future user > of this API is that it is going to be > succinct, > work as > efficiently as possible and be clearly > documented. > Whether > functions are overloaded or not is not really > crucial. If I > want to use the Web MIDI API, then I'm > going to > have to learn > it one way or the other. > > But I'd like to open another can of worms: :-) > The current version of the API [1] supports > MIDIMessages and > MIDIEvents. Would it be possible for it to > provide > Sequence > and Track objects as well? I've written my own > versions of > these for my current project (in > Javascript), but > think there > must be people out there who know much > better how > to write > them than I do. They are prime targets for > optimization. We > should not all have to re-invent the wheel. > Interestingly, there are versions of these > objects > in Leslie > Sanford's popular C# MIDI Toolkit [2]., so > he also > seems to > have thought that they were worth providing. > > Any thoughts? > > Best, > James > > [1] > https://dvcs.w3.org/hg/audio/raw-file/tip/midi/specification.html#callback-navigatormidiaccesssuccesscallback-parameters > [2] > http://www.codeproject.com/Articles/6228/C-MIDI-Toolkit > > > > > > > -- www.james-ingram-act-two.de > <http://www.james-ingram-act-two.de> > <http://www.james-ingram-act-two.de> > <http://www.james-ingram-act-two.de> > > > > > -- www.james-ingram-act-two.de > <http://www.james-ingram-act-two.de> > <http://www.james-ingram-act-two.de> > > > > > > -- > www.james-ingram-act-two.de <http://www.james-ingram-act-two.de> > > > -- www.james-ingram-act-two.de
Received on Wednesday, 12 September 2012 21:59:03 UTC