- From: James Ingram <j.ingram@netcologne.de>
- Date: Wed, 12 Sep 2012 19:10:38 +0200
- To: Chris Wilson <cwilso@google.com>
- CC: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>, public-audio@w3.org
Hi Chris, Jussi, all,
As you pointed out, this kind of thing may well belong in a standard
library, but I think it might be worth taking a closer look at the
problems that are going to have to be solved, in case we discover that
lower level solutions might be better after all.
As I said, I'm currently writing a sequencer in Javascript, using
JazzMIDIBridge [5] to emulate the future Web MIDI API.
I've created my own definition of a Sequence, and can now load such
Sequences from files. The immediate problem is now to write a simple
sequence.play()
function which will play the sequence, from beginning to end, as
accurately as possible.
In general, I expect sequences to be large (there should be no limit, in
principle, to their duration), and to contain lots of continuous control
data. So I'm creeping up on the problem, by writing a few test files.
(I'm working on Windows, and using the latest Chrome browser.)
Test 1 [1] : Starting with the JazzMIDIBridge example5 file (see [5]):
The original example5 file uses setTimeout() naively to simulate a
sequencer, and the first thing I wanted to do was implement the approach
described in [6] to cancel out the delays caused by other processes
running in the system. In other words, the delay passed to setTimeout()
is adjusted in this test so that
(systemTime + delay) is equal to the MIDIMessage.timestamp.
Having done that, I set the code up to play a simple sequence of 50
NOTE_ON messages at 200 milliseconds per message.
The result is much more accurate than it would have been without
adjusting the delay.
I'm not too worried about Date().getTime()'s (lack of) accuracy.
Presumably the Web Audio timer would be better, but the principle would
not change. My machine has no problem playing this test sequence
completely accurately (according to the system clock).
Test 2 [2] : the same code as test 1, except that it tries to play 1000
NOTE_ON messages at 100 milliseconds per message. That should take 100
seconds. As can be seen from the output, the delay value being passed to
setTimeout() tends to get smaller over time, because the amount of time
being used for other processes is accumulating. Eventually, the delay is
going to go negative, and it will no longer be possible to honour the
message's timestamp at all. That can be seen happening, on my machine,
in Test 3.
Test 3 [3]: is the same code as tests 1 & 2, but, with 2000 NOTE_ONs at
50 ms per message.
(I'm aware that there are issues connected with passing small delay
values to setTimeout().)
Test 4 [4]: This test uses 500 repeats of the sequence 50, 50, 50, 200
milliseconds.
I'm trying to play the sequence "as accurately as possible", and tests
1-3 show that no computer on earth is going to be able to play long,
dense sequences accurately according to the timestamps forever if it is
simultaneously doing other things. (Is that right?)
So I've started using "time drift" or "local time": The base time from
which the timestamps are measured is allowed to drift, and the drift
occurs where it is least noticed -- during the longer notes. Adding 10ms
to a long duration is less noticeable than adding it to a short one. I
don't really care if a performance which should last several minutes, is
actually a few seconds longer.
So the function which uses the timestamp to calculate the delay has to
use the previous timestamp as well, in order to know whether it can add
a bit of drift or not. The previous timestamp is therefore cached.
The algorithm for adding drift could probably be improved, but the fact
of having to add drift, and the algorithm's dependency on the durations
between timestamps is not going to change.
As defined, this sequence should should take exactly 175000 miliseconds
(just short of 3 minutes), but the last time I ran it it took about 4
seconds longer than that. But at least it sounded okay (if you like that
sort of thing) and got that far without wanting to go backwards in time! :-)
Am I missing something? I'm not at all sure that this is fundamentally
the right way to go. I was rather hoping that there could be accurate
API functions for playing/pausing/stopping/rewinding/ MIDI sequences.
But that would mean agreeing on Track and Sequence definitions. Which is
where I came in. :-)
all the best,
James
[1] http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence1.html
<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html>
[2] http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence2.html
<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html>
[3] http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence3.html
<http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html>
[4] http://james-ingram.de/tests/JazzMIDIBridge/ji-known-sequence4.html
[5] https://github.com/abudaan/JazzMIDIBridge
[6] http://www.sitepoint.com/creating-accurate-timers-in-javascript/
Am 12.09.2012 01:14, schrieb Chris Wilson:
> Hey James-
>
> Actually, sendMIDIMessage takes MIDIMessages, which can have a
> timestamp - it's not an "absolute" timestamp in the sense of being
> relative to the UNIX epoch or anything, but you can in fact send an
> entire buffer of MIDIMessages with their timestamps to be sent at the
> same time.
>
> This doesn't address wanting to change tempo dynamically (or pausing),
> of course; but you absolutely can do this. I'm not sure it's what
> you'd want in most cases (other than simple playback), but it is
> definitely there.
>
> On Tue, Sep 11, 2012 at 2:00 AM, James Ingram <j.ingram@netcologne.de
> <mailto:j.ingram@netcologne.de>> wrote:
>
> Hi Chris, Jussi, all,
>
> Thanks, Chris, for this answer. I could certainly live with using
> a dedicated Javascript library which provided Tracks and
> Sequences. Probably, some libraries will be more suitable for some
> tasks than others, and its best to leave that flexibility outside
> the MIDI API.
> And I take your point that the current API needs to get locked
> down before we start talking about v2 (libraries).
>
> But I'm still wondering if the API is really complete as it
> stands. I'm currently wrestling with the problem of accurate
> timing, and am asking myself why this is such a hassle. If there
> was an accurate
> MIDIOutput.sendAtAbsoluteTime(midiIMessage, absoluteTime)
> then the underlying code could take advantage of existing timing
> facilities, and the API's user (me) could get on with solving
> other problems. I suspect that timing problems in particular would
> be best solved at a low level...
>
> best,
> James
>
> Am 11.09.2012 00:48, schrieb Chris Wilson:
>
> Hey James -
>
> A while ago, I poked at implementing Standard MIDI File
> reading in Javascript
> (https://github.com/cwilso/Standard-MIDI-File-reader), and my
> conclusion was basically that it would be best provided as a
> library and/or as baked into the standard <audio> player
> engine in the browser, rather than centering the MIDI API on
> it. There are a lot of challenges with doing rewinding, for
> example, and seeking. There is also a lot of dynamic control
> hooking in to tempo changes, live information exposure of
> events, etc., that any "serious" sequencer would probably want
> to have. Even in that relatively naive reader, there are a
> bunch of different types of data - it has more exposure than
> the MIDI API does by itself. Once we've got the MIDI API
> locked down and implemented, I'll probably go back and flesh
> out my SMF reader as a library for reading and playing SMFs,
> and we can talk about if it's a useful v2 standard library.
>
> -C
>
> On Mon, Sep 10, 2012 at 5:06 AM, James Ingram
> <j.ingram@netcologne.de <mailto:j.ingram@netcologne.de>
> <mailto:j.ingram@netcologne.de
> <mailto:j.ingram@netcologne.de>>> wrote:
>
> Hi Jussi,
>
> I think support for Sequences would be useful (and easy to
> provide), even if we are not talking about file output.
> Loading sequences from files is very common in MIDI
> programs, and
> I see no reason why that should not be done on the web. So it
> would be nice to have a standard Sequence definition with
> which to
> work. I'm not saying that Sequences necessarily have to be
> loaded
> from SMFs, but if the Sequence definition existed, it could
> obviously be given a loadFromSMF(url) member function.
>
> Also, if we had a Sequence object, it would wrap the output
> device, and play(), stop(), rewind() etc. functions could be
> provided quite cheaply out of the box. That might make Web
> MIDI
> even more attractive. :-)
>
> But maybe I'm too close to the problem. Is there anyone
> else out
> there who needs to send a Sequence to the output device?
>
> best,
> James
>
>
> Hi James,
>
> A side note: the current spec has the notion of
> MIDIEvent as a
> DOM event that occurs when MIDIMessage(s) are received.
>
> The toolkit you're referring to deals both with MIDI
> files (SMF)
> and MIDI devices. Our API is designed to work with
> just devices,
> and MIDI devices are just I/O, there's no notion of
> high level
> concepts such as tracks (although there's channels,
> but that's
> already in the spec) or sequences, unlike SMF does. It
> would seem
> a bit weird to me to complicate the spec with such
> high-level
> concepts as we don't even support reading / writing
> SMF files,
> nor do I think we should as the intent of the device is
> explicitly to allow communication with MIDI devices.
>
> Cheers,
> Jussi
>
> On Mon, Sep 10, 2012 at 12:32 PM, James Ingram
> <j.ingram@netcologne.de
> <mailto:j.ingram@netcologne.de>
> <mailto:j.ingram@netcologne.de
> <mailto:j.ingram@netcologne.de>>> wrote:
>
> Hi Chris, all,
>
> Chris Wilson said (apropos Bug 18764)
>
> I'd like to explicitly ask for MORE feedback
> from developers
> using MIDI today on this issue.
>
> I'm following the Bug 18764 debate very closely,
> but don't
> have the in-depth knowledge of MIDI's finer points to
> contribute much to the subject. All I want as a
> future user
> of this API is that it is going to be succinct,
> work as
> efficiently as possible and be clearly documented.
> Whether
> functions are overloaded or not is not really
> crucial. If I
> want to use the Web MIDI API, then I'm going to
> have to learn
> it one way or the other.
>
> But I'd like to open another can of worms: :-)
> The current version of the API [1] supports
> MIDIMessages and
> MIDIEvents. Would it be possible for it to provide
> Sequence
> and Track objects as well? I've written my own
> versions of
> these for my current project (in Javascript), but
> think there
> must be people out there who know much better how
> to write
> them than I do. They are prime targets for
> optimization. We
> should not all have to re-invent the wheel.
> Interestingly, there are versions of these objects
> in Leslie
> Sanford's popular C# MIDI Toolkit [2]., so he also
> seems to
> have thought that they were worth providing.
>
> Any thoughts?
>
> Best,
> James
>
> [1]
> https://dvcs.w3.org/hg/audio/raw-file/tip/midi/specification.html#callback-navigatormidiaccesssuccesscallback-parameters
> [2]
> http://www.codeproject.com/Articles/6228/C-MIDI-Toolkit
>
>
>
>
>
>
> -- www.james-ingram-act-two.de
> <http://www.james-ingram-act-two.de>
> <http://www.james-ingram-act-two.de>
>
>
>
>
> --
> www.james-ingram-act-two.de <http://www.james-ingram-act-two.de>
>
>
>
--
www.james-ingram-act-two.de
Received on Wednesday, 12 September 2012 17:11:23 UTC