Re: Graphical Scores and SVG

Hi Joe,
After reading your email, I personally like approach B, because it offers
flexibility to synchronize the score to multiple media formats (even
video), not just MIDI.

That being said, I also strongly agree that this area is not a priority at
the moment. Getting the initial CWMN semantics down should be our focus;
it's complex enough!

Thank you for all your work. I am really extremely pleased with how this
process is going. Keep up the good work!

J. Sawruk

On Wed, Mar 29, 2017 at 11:58 AM, Joe Berkovitz <joe@noteflight.com> wrote:

> Hi all,
>
> I think this is a good moment to talk in more detail about the potential
> uses of SVG in the work of this group since we've had a fair bit of
> activity on that topic.
>
> To that end, I've just had a very useful exchange with W3C's Chris Lilley,
> who is copied on this email. Chris is a computer scientist who originally
> chaired the W3C SVG Working Group when it began in 1998, and saw SVG
> through all the way from an abstract idea to its modern realization. So
> it's fair to say he's been observing and thinking about its uses for a very
> long time, with an expert's perspective. Chris is also a member of the W3C
> Web Audio Working Group which is responsible for both audio and MIDI
> standards on the Web.
>
> What I'll present here is my current thinking, informed by some thoughtful
> points made by Chris -- who I'm encouraging to jump in with his own words.
>
> Let me say first that I see at least two potential uses for SVG in our
> community group, and they seem to harmonize perfectly:
>
> 1. SVG (with some relationship to sound) can represent musical scores with
> arbitrary visual and sonic content. Thanks to James Ingram for highlighting
> this particular case to the group.
>
> 2. SVG (with some relationship to sound) can serve as an intermediate
> format that represents *a specific rendering* of semantic music notation
> markup such as MNX, MusicXML, MEI, or anything else.
>
> So far a lot of discussion has revolved around #1, but #2 is at least as
> significant. Consider that it permits semantic markup to be converted into
> a format that can be easily displayed and played back by an application
> that is much simpler than one that processes semantic markup. Of course,
> that format is necessarily quite limited in other ways (try transposing it
> to another key, or reformatting it for a different display size!)  But, as
> a final-delivery mechanism, #2 seems to have a lot of merit. It could
> provide a standardized way to package the output of notation renderers,
> regardless of what markup language they use. In fact, MathML (a semantic
> markup language for math formulas) is routinely converted to SVG by various
> standard libraries for exactly this purpose.
>
> Now: I believe we don't need to get into a big debate about which use is
> more important. They both are. Also, in neither case do they eliminate our
> need for a semantic markup language within the confines of some known
> musical idiom, so there's no need to stop that train from leaving the
> station. MNX explicitly makes room for graphical encodings to be placed
> within it.
>
> Relative to SVG, then, the key question is: What's the best way to augment
> an SVG document with information on sonic performance? There are multiple
> ways to do it. Chris and I discussed several very high-level approaches:
>
> A. Intersperse performance data (e.g. individual MIDI or note events)
> throughout an SVG file. James's proposed approach follows this pattern:
> MIDI codes are sprinkled directly into the file and attached to graphical
> elements. One could also use a different means to specify events, say like
> the way that MNX proposes.
>
> B. Intersperse *references* to a separate performance file (e.g. a
> Standard MIDI file, MP3 audio file) throughout an SVG file. In this
> approach, SVG elements are tagged with simpler data that "points to" some
> time or time range within a separate document. MEI uses this technique in
> places. Example:
>     <measure sound-ref="ThisPiece.midi" sound-start="1:22"
> sound-end="1:24">...
>
> C. Create a completely separate mapping file that identifies a
> correspondence between SVG and a performance file. Such a file might
> contain entries like this:
>     <measure-mapping graphics-ref="ThisPiece.svg#m21"
> sound-ref="ThisPiece.midi" sound-start="1:22" sound-end="1:24"/>
>
> I do not think there is a clear winner among these, and I don't think we
> should immediately get into the weeds. The next step in this discussion --
> when we have it -- is to look at the pros and cons of these various
> approaches for uniting graphical and sonic information. Each has advantages
> and disadvantages, and they need to be brought to the surface in what will
> be a lengthy discussion of its own. All of the above techniques have been
> tried in different contexts and there are definite lessons to be learned.
>
> As a corollary: let's stop debating the importance of pure graphical music
> encoding. There is no need for a debate: we agree that it *is* important.
> However, its role and its structure do need not to be settled in advance of
> our work to move ahead on CWMN semantic music encoding. We will need time
> to tease out the answers to the questions raised above.
>
> Finally a word on Frankfurt: the co-chairs plan to devote a limited period
> of time to discussing this topic, but it will certainly be smaller than
> many would like (myself included). We are limited by the other big things
> on the agenda. But, in truth, most of the good stuff in standards groups,
> happens on email and Github over time, not in large in-person meetings!
>
> Best,
> .            .       .    .  . ...Joe
>
> Joe Berkovitz
> Founder
> Noteflight LLC
>
> 49R Day Street
> Somerville MA 02144
> USA
>
> "Bring music to life"
> www.noteflight.com
>