- From: Chris Wilson <cwilso@google.com>
- Date: Wed, 12 Sep 2012 10:47:03 -0700
- To: James Ingram <j.ingram@netcologne.de>
- Cc: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, public-audio@w3.org
- Message-ID: <CAJK2wqXUxY010GBNpVd7rH1nkQ+vtc+ZhbZdDqYFBNMz3jbEMg@mail.gmail.com>
James, I'm quite confused. sendTimestampedMIDIMessage in that shim is not a function you should have to write or edit; the Web MIDI API has a "send this message, at this absolute time" capability. You shouldn't need to deal with clock drift; in a real, native implementation of the Web MIDI API, a lower-level timing system would be handling this, not Javascript's setTimeout/setInterval; much like the Web Audio API has its own system for scheduling. This is going to be a weak point in any polyfill (the one you point to, and the one I'm writing) - you simply cannot get access to a high-enough resolution clock to properly implement the timestamps, and none of the current ways to access MIDI in Javascript that I know of expose a higher-res timing system. For playing/pausing/stopping, the only thing I think we may be missing is the ability to cancel all scheduled MIDI messages, although before imposing that I need to look more closely at how the underlying systems support such a beast; you may need to close the device in order to clear it. My suggested implementation tactic would be to use setInterval to pre-schedule MIDI events in the send message queue, keeping some arbitrary interval of time (a tenth of a second or so?) ahead of the current DOMHRT time; this slower loop can then be cancelled whenever you hit stop. On Wed, Sep 12, 2012 at 10:10 AM, James Ingram <j.ingram@netcologne.de>wrote: > > Hi Chris, Jussi, all, > > As you pointed out, this kind of thing may well belong in a standard > library, but I think it might be worth taking a closer look at the problems > that are going to have to be solved, in case we discover that lower level > solutions might be better after all. > > As I said, I'm currently writing a sequencer in Javascript, using > JazzMIDIBridge [5] to emulate the future Web MIDI API. > > I've created my own definition of a Sequence, and can now load such > Sequences from files. The immediate problem is now to write a simple > sequence.play() > function which will play the sequence, from beginning to end, as > accurately as possible. > > In general, I expect sequences to be large (there should be no limit, in > principle, to their duration), and to contain lots of continuous control > data. So I'm creeping up on the problem, by writing a few test files. (I'm > working on Windows, and using the latest Chrome browser.) > > Test 1 [1] : Starting with the JazzMIDIBridge example5 file (see [5]): > The original example5 file uses setTimeout() naively to simulate a > sequencer, and the first thing I wanted to do was implement the approach > described in [6] to cancel out the delays caused by other processes running > in the system. In other words, the delay passed to setTimeout() is adjusted > in this test so that > (systemTime + delay) is equal to the MIDIMessage.timestamp. > Having done that, I set the code up to play a simple sequence of 50 > NOTE_ON messages at 200 milliseconds per message. > The result is much more accurate than it would have been without adjusting > the delay. > I'm not too worried about Date().getTime()'s (lack of) accuracy. > Presumably the Web Audio timer would be better, but the principle would not > change. My machine has no problem playing this test sequence completely > accurately (according to the system clock). > > Test 2 [2] : the same code as test 1, except that it tries to play 1000 > NOTE_ON messages at 100 milliseconds per message. That should take 100 > seconds. As can be seen from the output, the delay value being passed to > setTimeout() tends to get smaller over time, because the amount of time > being used for other processes is accumulating. Eventually, the delay is > going to go negative, and it will no longer be possible to honour the > message's timestamp at all. That can be seen happening, on my machine, in > Test 3. > > Test 3 [3]: is the same code as tests 1 & 2, but, with 2000 NOTE_ONs at 50 > ms per message. > (I'm aware that there are issues connected with passing small delay values > to setTimeout().) > > Test 4 [4]: This test uses 500 repeats of the sequence 50, 50, 50, 200 > milliseconds. > I'm trying to play the sequence "as accurately as possible", and tests 1-3 > show that no computer on earth is going to be able to play long, dense > sequences accurately according to the timestamps forever if it is > simultaneously doing other things. (Is that right?) > So I've started using "time drift" or "local time": The base time from > which the timestamps are measured is allowed to drift, and the drift occurs > where it is least noticed -- during the longer notes. Adding 10ms to a long > duration is less noticeable than adding it to a short one. I don't really > care if a performance which should last several minutes, is actually a few > seconds longer. > So the function which uses the timestamp to calculate the delay has to use > the previous timestamp as well, in order to know whether it can add a bit > of drift or not. The previous timestamp is therefore cached. > The algorithm for adding drift could probably be improved, but the fact of > having to add drift, and the algorithm's dependency on the durations > between timestamps is not going to change. > As defined, this sequence should should take exactly 175000 miliseconds > (just short of 3 minutes), but the last time I ran it it took about 4 > seconds longer than that. But at least it sounded okay (if you like that > sort of thing) and got that far without wanting to go backwards in time! :-) > > Am I missing something? I'm not at all sure that this is fundamentally the > right way to go. I was rather hoping that there could be accurate API > functions for playing/pausing/stopping/**rewinding/ MIDI sequences. But > that would mean agreeing on Track and Sequence definitions. Which is where > I came in. :-) > > all the best, > James > > [1] http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-** > sequence1.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence1.html>< > http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-**sequence4.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > > > [2] http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-** > sequence2.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence2.html>< > http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-**sequence4.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > > > [3] http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-** > sequence3.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence3.html>< > http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-**sequence4.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > > > [4] http://james-ingram.de/tests/**JazzMIDIBridge/ji-known-** > sequence4.html<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html> > > [5] https://github.com/abudaan/**JazzMIDIBridge<https://github.com/abudaan/JazzMIDIBridge> > [6] http://www.sitepoint.com/**creating-accurate-timers-in-**javascript/<http://www.sitepoint.com/creating-accurate-timers-in-javascript/> > > > Am 12.09.2012 01:14, schrieb Chris Wilson: > >> Hey James- >> >> Actually, sendMIDIMessage takes MIDIMessages, which can have a timestamp >> - it's not an "absolute" timestamp in the sense of being relative to the >> UNIX epoch or anything, but you can in fact send an entire buffer of >> MIDIMessages with their timestamps to be sent at the same time. >> >> This doesn't address wanting to change tempo dynamically (or pausing), of >> course; but you absolutely can do this. I'm not sure it's what you'd want >> in most cases (other than simple playback), but it is definitely there. >> >> On Tue, Sep 11, 2012 at 2:00 AM, James Ingram <j.ingram@netcologne.de<mailto: >> j.ingram@netcologne.de**>> wrote: >> >> Hi Chris, Jussi, all, >> >> Thanks, Chris, for this answer. I could certainly live with using >> a dedicated Javascript library which provided Tracks and >> Sequences. Probably, some libraries will be more suitable for some >> tasks than others, and its best to leave that flexibility outside >> the MIDI API. >> And I take your point that the current API needs to get locked >> down before we start talking about v2 (libraries). >> >> But I'm still wondering if the API is really complete as it >> stands. I'm currently wrestling with the problem of accurate >> timing, and am asking myself why this is such a hassle. If there >> was an accurate >> MIDIOutput.sendAtAbsoluteTime(**midiIMessage, absoluteTime) >> then the underlying code could take advantage of existing timing >> facilities, and the API's user (me) could get on with solving >> other problems. I suspect that timing problems in particular would >> be best solved at a low level... >> >> best, >> James >> >> Am 11.09.2012 00:48, schrieb Chris Wilson: >> >> Hey James - >> >> A while ago, I poked at implementing Standard MIDI File >> reading in Javascript >> (https://github.com/cwilso/**Standard-MIDI-File-reader<https://github.com/cwilso/Standard-MIDI-File-reader>), >> and my >> conclusion was basically that it would be best provided as a >> library and/or as baked into the standard <audio> player >> engine in the browser, rather than centering the MIDI API on >> it. There are a lot of challenges with doing rewinding, for >> example, and seeking. There is also a lot of dynamic control >> hooking in to tempo changes, live information exposure of >> events, etc., that any "serious" sequencer would probably want >> to have. Even in that relatively naive reader, there are a >> bunch of different types of data - it has more exposure than >> the MIDI API does by itself. Once we've got the MIDI API >> locked down and implemented, I'll probably go back and flesh >> out my SMF reader as a library for reading and playing SMFs, >> and we can talk about if it's a useful v2 standard library. >> >> -C >> >> On Mon, Sep 10, 2012 at 5:06 AM, James Ingram >> <j.ingram@netcologne.de <mailto:j.ingram@netcologne.de**> >> <mailto:j.ingram@netcologne.de >> >> <mailto:j.ingram@netcologne.de**>>> wrote: >> >> Hi Jussi, >> >> I think support for Sequences would be useful (and easy to >> provide), even if we are not talking about file output. >> Loading sequences from files is very common in MIDI >> programs, and >> I see no reason why that should not be done on the web. So it >> would be nice to have a standard Sequence definition with >> which to >> work. I'm not saying that Sequences necessarily have to be >> loaded >> from SMFs, but if the Sequence definition existed, it could >> obviously be given a loadFromSMF(url) member function. >> >> Also, if we had a Sequence object, it would wrap the output >> device, and play(), stop(), rewind() etc. functions could be >> provided quite cheaply out of the box. That might make Web >> MIDI >> even more attractive. :-) >> >> But maybe I'm too close to the problem. Is there anyone >> else out >> there who needs to send a Sequence to the output device? >> >> best, >> James >> >> >> Hi James, >> >> A side note: the current spec has the notion of >> MIDIEvent as a >> DOM event that occurs when MIDIMessage(s) are received. >> >> The toolkit you're referring to deals both with MIDI >> files (SMF) >> and MIDI devices. Our API is designed to work with >> just devices, >> and MIDI devices are just I/O, there's no notion of >> high level >> concepts such as tracks (although there's channels, >> but that's >> already in the spec) or sequences, unlike SMF does. It >> would seem >> a bit weird to me to complicate the spec with such >> high-level >> concepts as we don't even support reading / writing >> SMF files, >> nor do I think we should as the intent of the device is >> explicitly to allow communication with MIDI devices. >> >> Cheers, >> Jussi >> >> On Mon, Sep 10, 2012 at 12:32 PM, James Ingram >> <j.ingram@netcologne.de >> <mailto:j.ingram@netcologne.de**> >> <mailto:j.ingram@netcologne.de >> >> <mailto:j.ingram@netcologne.de**>>> wrote: >> >> Hi Chris, all, >> >> Chris Wilson said (apropos Bug 18764) >> >> I'd like to explicitly ask for MORE feedback >> from developers >> using MIDI today on this issue. >> >> I'm following the Bug 18764 debate very closely, >> but don't >> have the in-depth knowledge of MIDI's finer points to >> contribute much to the subject. All I want as a >> future user >> of this API is that it is going to be succinct, >> work as >> efficiently as possible and be clearly documented. >> Whether >> functions are overloaded or not is not really >> crucial. If I >> want to use the Web MIDI API, then I'm going to >> have to learn >> it one way or the other. >> >> But I'd like to open another can of worms: :-) >> The current version of the API [1] supports >> MIDIMessages and >> MIDIEvents. Would it be possible for it to provide >> Sequence >> and Track objects as well? I've written my own >> versions of >> these for my current project (in Javascript), but >> think there >> must be people out there who know much better how >> to write >> them than I do. They are prime targets for >> optimization. We >> should not all have to re-invent the wheel. >> Interestingly, there are versions of these objects >> in Leslie >> Sanford's popular C# MIDI Toolkit [2]., so he also >> seems to >> have thought that they were worth providing. >> >> Any thoughts? >> >> Best, >> James >> >> [1] >> https://dvcs.w3.org/hg/audio/**raw-file/tip/midi/** >> specification.html#callback-**navigatormidiaccesssuccesscall** >> back-parameters<https://dvcs.w3.org/hg/audio/raw-file/tip/midi/specification.html#callback-navigatormidiaccesssuccesscallback-parameters> >> [2] >> http://www.codeproject.com/**Articles/6228/C-MIDI-Toolkit<http://www.codeproject.com/Articles/6228/C-MIDI-Toolkit> >> >> >> >> >> >> >> -- www.james-ingram-act-two.de >> <http://www.james-ingram-act-**two.de<http://www.james-ingram-act-two.de> >> > >> <http://www.james-ingram-act-**two.de<http://www.james-ingram-act-two.de> >> > >> >> >> >> >> -- www.james-ingram-act-two.de <http://www.james-ingram-act-** >> two.de <http://www.james-ingram-act-two.de>> >> >> >> >> > > -- > www.james-ingram-act-two.de > > >
Received on Wednesday, 12 September 2012 17:47:38 UTC