Re: Music Notation on the Web

Hi Tom,

I'm sure we both agree that MIDI is an essential and widely adopted  
standard, and that that it serves an admirable purpose in the music  
world in capturing performance data that is playable on nearly every  
device or platform.

However, communicating the information found in a notated score has  
never been a primary goal of MIDI.  Thus, it's not a criticism of MIDI  
to observe that it does not perform  very well as a music notation  
format except in some very simple cases.  MIDI has a few optional  
features bolted on that capture things like key and time signatures,  
but otherwise it does not communicate the same kind of information  
that is found in a score file.  I think this is fine: it was not  
intended to do so, and it would be a much worse performance data  
format if it tried to cover this base.

Consequently, when Sibelius, Finale, Noteflight or any other notation  
program imports MIDI, the application must invent from whole cloth  
much of the notational data in the resulting score.  Much of that  
invented information is musically incorrect, and amounts to the best  
possible assumption that can be made without a lot of missing  
information.

A few salient examples:

- MIDI does not distinguish between enharmonic notes, such as D# and  
Eb. In a score, and to a performer, these are very different. The key  
signature doesn't suffice to figure these out; only the composer  
really knows.
- MIDI does not distinguish between different ways of writing the same  
rhythm, such as a quarter tied to an eighth versus a dotted quarter.  
These are likewise very important to performers.
- MIDI does not distinguish articulatory variants that look identical  
in performance data, say, between a staccato quarter note and a full- 
value 16th note. On many instruments these yield completely different  
sounds in live performance.
- MIDI does not distinguish between notes in different staves or  
voices of the same instrumental part, which are essential to correct  
interpretation.

That list could be a lot longer if we include all the various types of  
expressive markings that are an important part of score writing and  
which capture the composer's intent.

Importing MIDI is thus one of the most difficult tasks faced by a  
music notation developer, since the code must synthesize reasonable  
approaches in these cases and numerous others. The best one can hope  
for is "reasonable" output for very simple music.  For more complex  
music, the resulting score is usually not acceptable at all and  
require a lot of manual fixup after conversion from MIDI.

Best,

... .  .    .       Joe

Joe Berkovitz
President
Noteflight LLC
84 Hamilton St, Cambridge, MA 02139
phone: +1 978 314 6271
www.noteflight.com

On Dec 12, 2010, at 1:59 AM, Tom White (MMA) wrote:

> But "Standard MIDI Files" store pitch and rhythm information for  
> every note in a musical performance, from which it is possible for  
> computer programs (including Sibelius and Finale) to create a  
> musical score (one that is lacking in some formatting)... is that  
> not what you are looking for?
>
> Tom White
> MMA
>
> From: Cutler, Roger (RogerCutler) [mailto:RogerCutler@chevron.com]
> Sent: Saturday, December 11, 2010 3:56 PM
> To: Tom White (MMA); public-xg-audio@w3.org
> Subject: RE: Music Notation on the Web
>
> As Mr. Berkovitz says, Midi is pure performance data for electronic  
> instruments.  Quoting from Wikipedia, “it sends event messages about  
> pitch and intensity, control signals for parameters such as volume,  
> vibrato and panning, cues, and clock signals to set the tempo.”   
> That’s very different from a music score and serves a different  
> purpose.
>
> From: public-xg-audio-request@w3.org [mailto:public-xg-audio-request@w3.org 
> ] On Behalf Of Tom White (MMA)
> Sent: Saturday, December 11, 2010 4:42 PM
> To: public-xg-audio@w3.org
> Subject: RE: Music Notation on the Web
>
> Roger,
>
> You say you are interested in exchanging the "notes in the proper  
> rhythms" and doing the "formatting in the new environment"... and  
> you want to "import the notes into Sibelius and Finale and handle  
> the formatting there".
>
> Is there some reason you can't use MIDI?
>
> Tom White
> MMA
>
> From: public-xg-audio-request@w3.org [mailto:public-xg-audio-request@w3.org 
> ] On Behalf Of Cutler, Roger (RogerCutler)
> Sent: Saturday, December 11, 2010 12:54 PM
> To: Joseph Berkovitz
> Cc: public-xg-audio@w3.org
> Subject: RE: Music Notation on the Web
>
> I may be demonstrating my ignorance here, but I have not personally  
> had good experiences with MusicXML.  I recall that I tried to use  
> this format to transfer some music between Finale and Sibelius (I’ve  
> forgotten which way, but probably toward Sibelius) and I had a LOT  
> of trouble.  The bottom line was that it basically did not work.  It  
> is my impression that neither Finale nor Sibelius support MusicXML  
> in a very complete fashion.  I also think (but am not positive) that  
> MusicXML attempts to model features of the music that are involved  
> with formatting as well as the content of the music.  In fact, I  
> think, but am not sure, that this was probably the source of the  
> problems I had using it.  I would have been perfectly happy to start  
> with a bare bones version of the music (that is, the notes in the  
> proper rhythms) and do the formatting in the new environment, but I  
> seem to recall that it tried to pull across a bunch of very complex  
> stuff associated with formatting that didn’t really work and got in  
> the way of using the information at all.
>
> I believe that there are other possible objections to using MusicXML  
> as a starting point:
>
> 1.       One of the responses I got at the TPAC was a very strong  
> opinion, which I initially was skeptical of but eventually found  
> convincing, that a music representation scheme used on the Web  
> should be capable of practically supporting use by hand for simple  
> tasks.  ABC is an example of a markup for music content that has  
> this character.  My impression is that MusicXML does not in that it  
> is, in all representations, quite complex and bulky.  Note that the  
> illustration shown by Hakon Lie at the TPAC of a music notation  
> markup being used in HTML 5 used ABC.
> 2.       My impression (again, perhaps showing my ignorance) is that  
> the takeup of MusicXML on the Web has been extremely limited.   
> Certainly none of the sites that I go to in order to get music offer  
> anything in MusicXML.  The most common offering is a PDF file, which  
> has obvious limitations.  What I REALLY want is a format that allows  
> me to import the notes into Sibelius or Finale and handle the  
> formatting there.
> 3.       I speculate that it might be easier to get whole-hearted  
> participation from the majors (Finale and Sibelius) – and my  
> impression is that their implementations of MusicXML are either not  
> whole-hearted or that it is extremely difficult – if the markup  
> standard confined itself to the CONTENT of the music and not the  
> formatting.  I say this because I get the impression that these  
> vendors consider their competitive advantage to involve formatting  
> AND because I also get the impression that some of the formatting  
> they do is extremely complex and difficult to represent as anything  
> but an image.
>
> In summary, it seem plausible to me that if one looks hard at the  
> requirements for a Web music notation that there may be no obvious  
> solution on the ground, or there is that it is not the “obvious”  
> solutions of either MusicXML or the ISO standard SMDL, which was  
> suggested as “obvious” by someone else.  And if indeed there is an  
> “obvious” solution – then what can be done to get people actually to  
> USE it????
>
> From: Joseph Berkovitz [mailto:joe@noteflight.com]
> Sent: Friday, December 10, 2010 8:20 PM
> To: Cutler, Roger (RogerCutler)
> Cc: public-xg-audio@w3.org
> Subject: Re: Music Notation on the Web
>
> Hi folks,
>
> As a creator of web-based music notation editing software, I welcome  
> a discussion on this topic, but with a large caution on its breadth  
> and depth. Semantic representation of music is such a complex area  
> that it's very likely to occupy the full bandwidth of any group that  
> takes up the challenge, and then some. I believe that music  
> notation, if addressed by the W3C, will almost certainly consume the  
> full attention of a single XG or WG.  I would of course be very  
> pleased to participate in the discussion or in groups that spin off.
>
> It's also necessary to mention that my fellow Audio XG member  
> Michael Good has devoted much of his career to successfully  
> developing and promulgating the MusicXML standard in industry and  
> academia. There are few people as qualified than Michael to be part  
> of this discussion. So there are at least two folks on this group  
> with a strong interest, and there may well be more.
>
> Now, as Michael stated in his response, that there is already a  
> widely adopted standard in the notation world, namely MusicXML which  
> his company Recordare owns. It is the de facto interchange standard  
> today for most music notation applications.  One might even take the  
> position that MusicXML's status on the ground in the community is  
> such that a separate W3C standards effort is unnecessary.  
> Personally, I would not go that far: my opinion is that a W3C  
> standard for music notation could possibly be a good thing, but that  
> such a standard would do very well to look at MusicXML as a starting  
> point (and as a potential ending point also, in that W3C could bless  
> MusicXML as the standard itself and carry its evolution forward).  
> Whatever the potential of a new standard might be, MusicXML reflects  
> a lot of information and wisdom accumulated over its lifetime and is  
> pretty well burned into the ecosystem today.
>
> Insofar as this group is concerned, I would very much like to see  
> the work on the Audio API be completed before commencing a challenge  
> as substantial as music notation. It would be great to start to  
> discuss it, but we should all be aware that some of the ratholes  
> will be miles deep, and the implementation mountain will be miles  
> high. Let me explain further by way of commenting on a few points  
> from the thread that Roger just posted:
>
> Wouldn't it be nice to be able to publish music in a way that the  
> CONTENT, as opposed to the formatting, could be picked up and used?   
> Isn't that kind of in the spirit of the Web?  And isn't it also kind  
> of in the spirit of        the Web to worry about the content before  
> you start messing around with binary streams and images?
>
> I share this enthusiasm for representing semantic musical content on  
> the Web -- it's my mission too, and what my company is all about.  
> However, it is not the case that binary audio streams are  
> necessarily a "formatting" of such content, any more than pixels are  
> necessarily "formatted text".  Audio is a primary medium, much of  
> whose content can not be derived from semantic notation by either a  
> human or a machine. Much music is never notated in the first place.  
> Although I am a musician with a traditional conservatory background,  
> I strongly believe that supporting pure sound generation on the Web  
> is a valid and essential musical enterprise in and of itself.   
> Programmatic audio and music notation content are complementary, and  
> it's not at all a cart-before-the-horse situation.
>
> Furthermore, audio support will help music notation take many  
> important steps towards becoming a first-class citizen on the web.  
> While music notation viewing could (with considerable effort) become  
> somewhat standardized in browsers, music notation editing is likely  
> to remain a diverse space with many disparate approaches. Any Web- 
> based notation editor will absolutely require good programmatic  
> audio support in order to be at all functional (and one of my  
> primary motivations in working with the Audio group is to allow the  
> Web to support such editors).
>
> [NOTE – There ARE companies involved with music notation and  
> publishing, but they are not in the W3C.  I’d really like to reach  
> out to them and try to involve them in this effort, and I have some  
> ideas how to make that attractive, or at least how NOT to make it  
> UNattractive – Roger]
>
> At least *some* are in the W3C, already ;)
>
> Getting support for maths in Web content was similarly backed by
> centuries of cultural heritage; but that alone wasn't enough to make
> it a major priority for all browser-makers... [snip]
>
> The effort involved in properly rendering even a basic subset of  
> conventional Western music notation is unreasonably large, for a  
> number of issues that aren't appropriate to go into here.  Music  
> notation has accreted from centuries' worth of experimentation  
> across various cultural shifts, shaped by the quirks of human visual  
> cognition. Due to the implementation effort, I share this concern  
> that browser-makers might simply ignore this area, unless there's a  
> strong case that music notation on the Web will have a broad  
> audience comparable to that for the other modern browser feature sets.
>
> On the whole I'd like to see a Web standard for music notation  
> emerge, when it can do so successfully with strong backing from the  
> industry and from the W3C.  Development of such a standard would  
> probably fertilize the software/music ecosystem for a very long time  
> to come, if the standard achieves a high level of acceptance. Let's  
> just proceed carefully and thoughtfully, with respect for the effort  
> level required, and the history of past efforts, and do it right.
>
> best,
>
> ... .  .    .       Joe
>
> Joe Berkovitz
> President
> Noteflight LLC
> 84 Hamilton St, Cambridge, MA 02139
> phone: +1 978 314 6271
> www.noteflight.com
>
>
>
>
>
>