Re: Music Notation on the Web

Roger,

As a notation vendor who supports MusicXML interchange, I'd be the  
first to say that document interchange experiences between notation  
apps are not at all what they should be. Negative experiences are,  
sadly, far from unusual (although "did not work" is a pretty rare  
outcome these days). It's worth looking at where these problems  
actually come from, however. While MusicXML could be improved in a  
number of ways, it's not fair to lay all these issues at its door.

(By the way, I haven't mentioned MIDI because it represents pure  
performance data rather than notation. Performance data and MIDI are  
incredibly relevant to this group, however, and certainly should be  
discussed.  I think that's a separate discussion though and MIDI has  
come up a bunch on this list.)

Desktop notation vendors have often failed to do a good job of  
supporting MusicXML. This has a great deal to do with the business  
dynamics of the notation software market, which sees much less  
cooperation than users might wish for. Given some of the desktop  
vendors' apparent lack of interest in allowing documents to migrate  
freely, it isn't clear that any standard would have fared as well as  
it should have.  But now, with music notation applications running  
directly in the browser (such as Noteflight and various HTML5 music  
rendering prototypes), I expect these dynamics to change for good.

Before responding to your other objections, let me clarify what I mean  
by taking MusicXML "as a starting point".  MusicXML's feature set can  
be seen as encoding a set of requirements that arose organically from  
diverse real-world needs. This is the starting point that I am  
referring to: the requirements alone. The details of MusicXML complex  
syntax and vendors' varying approaches to dealing with it are, at  
least at an initial stage of discussion, secondary.  If a solid next- 
generation standard is to be built, the requirements and use cases are  
the right place to start. I think that looked at in this way, MusicXML  
is a very valuable repository of ideas and concepts.  For a web  
standard, some of these may be relevant and some not. But starting  
from scratch risks overlooking all the thinking that has led to  
MusicXML, whatever its flaws might be.

>
> 1.       One of the responses I got at the TPAC was a very strong  
> opinion, which I initially was skeptical of but eventually found  
> convincing, that a music representation scheme used on the Web  
> should be capable of practically supporting use by hand for simple  
> tasks.  ABC is an example of a markup for music content that has  
> this character.  My impression is that MusicXML does not in that it  
> is, in all representations, quite complex and bulky.  Note that the  
> illustration shown by Hakon Lie at the TPAC of a music notation  
> markup being used in HTML 5 used ABC.

If we chose to adopt a requirement for ease of by-hand use, one could  
certainly simplify or rework some aspects of MusicXML syntax.  (I am  
not going to bother an audio list with gory details of beat divisions  
and time cursors.) I would not suggest paring it back as far as ABC,  
however, which I believe was designed primarily for by-hand use and is  
very limited in expressiveness.  Such changes, if they were to take  
place, would be significant but I believe they would have a small  
footprint in the schema.

> 2.       My impression (again, perhaps showing my ignorance) is that  
> the takeup of MusicXML on the Web has been extremely limited.   
> Certainly none of the sites that I go to in order to get music offer  
> anything in MusicXML.  The most common offering is a PDF file, which  
> has obvious limitations.  What I REALLY want is a format that allows  
> me to import the notes into Sibelius or Finale and handle the  
> formatting there.

In my view this problem has much to do with tepid vendor support for  
standards, as well as the historical lack of availability of free,  
high-quality notation editors and renderers (which is thankfully a  
thing of the past). It is not the MusicXML standard's fault per se.

> 3.       I speculate that it might be easier to get whole-hearted  
> participation from the majors (Finale and Sibelius) – and my  
> impression is that their implementations of MusicXML are either not  
> whole-hearted or that it is extremely difficult – if the markup  
> standard confined itself to the CONTENT of the music and not the  
> formatting.  I say this because I get the impression that these  
> vendors consider their competitive advantage to involve formatting  
> AND because I also get the impression that some of the formatting  
> they do is extremely complex and difficult to represent as anything  
> but an image.

First of all, a factual note: MusicXML treats formatting and  
performance information as optional. In my opinion a well-behaved  
application should not depend on the formatting and performance layers  
of MusicXML, and most applications avoid these obvious mistakes.

So, it's certainly an option to say that a hypothetical web standard  
based on MusicXML would leave out the formatting dimension, or supply  
it via an orthogonal stylesheet mechanism a la CSS. I would favor  
that.  The difficulty comes in the slippery slope of deciding just  
what amounts to "formatting".

The distinction between formatting and content is unusually hard to  
make with music notation. Some things (like the X offsets of various  
symbols) are very clearly formatting that is an artifact of some  
program's approach to music layout.  Other things, like the presence  
or absence of explicit accidentals, are considered to be much more  
than mere formatting by most musicians.  They are not required from a  
pure semantic point of view, in that they carry no obvious performance  
implication. But they do convey a compositional intent.

Is it impossible to work out these distinctions, filter out the  
formatting-oriented requirements, and come up with a primarily content- 
oriented standard?  No, it's not impossible -- it can definitely be  
done.  It's just going to be a lot of hard work to thrash it out.  
MusicXML hasn't had to cope with this question: it just makes many  
things optional, and it's in the eye of the producer/consumer whether  
they are "formatting" or "semantics".

> In summary, it seem plausible to me that if one looks hard at the  
> requirements for a Web music notation that there may be no obvious  
> solution on the ground, or there is that it is not the “obvious”  
> solutions of either MusicXML or the ISO standard SMDL, which was  
> suggested as “obvious” by someone else.  And if indeed there is an  
> “obvious” solution – then what can be done to get people actually to  
> USE it????

SMDL might have been obvious in 1996. It's an SGML/HyTime-era standard  
that has seen very little action in this century and almost no  
industry uptake.

Whatever the intrinsic design flaws of MusicXML today might be, my  
feeling is that the main blocking issues for use of the standard have  
not been the standard's fault.  This doesn't mean we can't improve on  
it, it just means we should be aware that industry dynamics have  
played a large role in this lack of adoption.  We are now on the way  
to notation editing and viewing becoming a Web-centric activity. This  
is perhaps the most important change of all, and one that will set the  
stage for a standard -- any decent one -- to be properly adopted and  
respected in the vendor and browser community.

Best,

... .  .    .       Joe

Joe Berkovitz
President
Noteflight LLC
84 Hamilton St, Cambridge, MA 02139
phone: +1 978 314 6271
www.noteflight.com






>
> From: Joseph Berkovitz [mailto:joe@noteflight.com]
> Sent: Friday, December 10, 2010 8:20 PM
> To: Cutler, Roger (RogerCutler)
> Cc: public-xg-audio@w3.org
> Subject: Re: Music Notation on the Web
>
> Hi folks,
>
> As a creator of web-based music notation editing software, I welcome  
> a discussion on this topic, but with a large caution on its breadth  
> and depth. Semantic representation of music is such a complex area  
> that it's very likely to occupy the full bandwidth of any group that  
> takes up the challenge, and then some. I believe that music  
> notation, if addressed by the W3C, will almost certainly consume the  
> full attention of a single XG or WG.  I would of course be very  
> pleased to participate in the discussion or in groups that spin off.
>
> It's also necessary to mention that my fellow Audio XG member  
> Michael Good has devoted much of his career to successfully  
> developing and promulgating the MusicXML standard in industry and  
> academia. There are few people as qualified than Michael to be part  
> of this discussion. So there are at least two folks on this group  
> with a strong interest, and there may well be more.
>
> Now, as Michael stated in his response, that there is already a  
> widely adopted standard in the notation world, namely MusicXML which  
> his company Recordare owns. It is the de facto interchange standard  
> today for most music notation applications.  One might even take the  
> position that MusicXML's status on the ground in the community is  
> such that a separate W3C standards effort is unnecessary.  
> Personally, I would not go that far: my opinion is that a W3C  
> standard for music notation could possibly be a good thing, but that  
> such a standard would do very well to look at MusicXML as a starting  
> point (and as a potential ending point also, in that W3C could bless  
> MusicXML as the standard itself and carry its evolution forward).  
> Whatever the potential of a new standard might be, MusicXML reflects  
> a lot of information and wisdom accumulated over its lifetime and is  
> pretty well burned into the ecosystem today.
>
> Insofar as this group is concerned, I would very much like to see  
> the work on the Audio API be completed before commencing a challenge  
> as substantial as music notation. It would be great to start to  
> discuss it, but we should all be aware that some of the ratholes  
> will be miles deep, and the implementation mountain will be miles  
> high. Let me explain further by way of commenting on a few points  
> from the thread that Roger just posted:
>
> Wouldn't it be nice to be able to publish music in a way that the  
> CONTENT, as opposed to the formatting, could be picked up and used?   
> Isn't that kind of in the spirit of the Web?  And isn't it also kind  
> of in the spirit of the Web to worry about the content before you  
> start messing around with binary streams and images?
>
> I share this enthusiasm for representing semantic musical content on  
> the Web -- it's my mission too, and what my company is all about.  
> However, it is not the case that binary audio streams are  
> necessarily a "formatting" of such content, any more than pixels are  
> necessarily "formatted text".  Audio is a primary medium, much of  
> whose content can not be derived from semantic notation by either a  
> human or a machine. Much music is never notated in the first place.  
> Although I am a musician with a traditional conservatory background,  
> I strongly believe that supporting pure sound generation on the Web  
> is a valid and essential musical enterprise in and of itself.   
> Programmatic audio and music notation content are complementary, and  
> it's not at all a cart-before-the-horse situation.
>
> Furthermore, audio support will help music notation take many  
> important steps towards becoming a first-class citizen on the web.  
> While music notation viewing could (with considerable effort) become  
> somewhat standardized in browsers, music notation editing is likely  
> to remain a diverse space with many disparate approaches. Any Web- 
> based notation editor will absolutely require good programmatic  
> audio support in order to be at all functional (and one of my  
> primary motivations in working with the Audio group is to allow the  
> Web to support such editors).
>
> [NOTE – There ARE companies involved with music notation and  
> publishing, but they are not in the W3C.  I’d really like to reach  
> out to them and try to involve them in this effort, and I have some  
> ideas how to make that attractive, or at least how NOT to make it  
> UNattractive – Roger]
>
> At least *some* are in the W3C, already ;)
>
> Getting support for maths in Web content was similarly backed by
> centuries of cultural heritage; but that alone wasn't enough to make
> it a major priority for all browser-makers... [snip]
>
> The effort involved in properly rendering even a basic subset of  
> conventional Western music notation is unreasonably large, for a  
> number of issues that aren't appropriate to go into here.  Music  
> notation has accreted from centuries' worth of experimentation  
> across various cultural shifts, shaped by the quirks of human visual  
> cognition. Due to the implementation effort, I share this concern  
> that browser-makers might simply ignore this area, unless there's a  
> strong case that music notation on the Web will have a broad  
> audience comparable to that for the other modern browser feature sets.
>
> On the whole I'd like to see a Web standard for music notation  
> emerge, when it can do so successfully with strong backing from the  
> industry and from the W3C.  Development of such a standard would  
> probably fertilize the software/music ecosystem for a very long time  
> to come, if the standard achieves a high level of acceptance. Let's  
> just proceed carefully and thoughtfully, with respect for the effort  
> level required, and the history of past efforts, and do it right.
>
> best,
>
> ... .  .    .       Joe
>
> Joe Berkovitz
> President
> Noteflight LLC
> 84 Hamilton St, Cambridge, MA 02139
> phone: +1 978 314 6271
> www.noteflight.com
>
>
>
>
>
>