Semantically enhanced SVG (proposal)

Hi,

I'd like to propose that the W3C Music Notation Community Group develop 
a semantically enhanced SVG file type for music scores.
_____________________________
Summary:

Semantically enhanced SVG (SeSVG) is a specialized SVG file type 
containing identifiable music notation objects. All such objects are 
defined using "class" attributes on the basic SVG elements. SeSVG files 
are just ordinary SVG files structured in a particular way, so it should 
be easy for applications that already export SVG to export SeSVG.
Interesting client applications can written because the contained 
objects are localized and identifiable, and the files can be reliably 
parsed.
Client applications will not only be browsers and SVG editors, but also 
web and desktop applications.
SeSVG is not competing with MusicXML (whose clients are music notation 
authoring tools).
The posting ends by describing how SeSVG files can be defined and managed.
_____________________________
Background:

Following the Community Group meeting in Frankfurt last April [1] and 
the subsequent email exchanges, we decided that everyone should go off 
and work on their own for a while. I've now made some progress that I'd 
like to share.

The main problem, for me, in previous discussions, was understanding 
what everyone meant by "semantic". For me, "semantic" refers both to the 
naming of symbols (in files and elsewhere) and to their meanings. Many 
problems disappear if these two concepts are kept strictly separate [2].

Another problem was that I have been developing a hybrid SVG format 
containing both ordinary SVG and MIDI information. In trying to explain 
that format, I was trying to tackle too many problems at once.
I now think that it would be a good idea to step back a bit, and 
approach the problem into two stages [3].
Interestingly, these two stages exactly reflect the distinction between 
symbol names and their meanings.

Stage 1, SeSVG (graphics, names): We should first define a standard way 
to write music graphics, independently of the meanings of the symbols. 
Since this will be a purely graphic standard, it can be expressed as an 
SVG specialisation. The standard should include both standard names for 
the symbol classes (e.g. "staff", "slur" etc.) and a description of how 
they nest. For more details of how this can be made to work, and work 
flexibly, see below.
Note that most music score authoring applications already export 
ordinary SVG (all of them can, in principle), so it should be fairly 
straightforward for them to adapt their code to export SeSVG. But note 
also, that the SeSVG standard has to be agreed before anyone can 
actually write an implementation that creates an instantiation. This 
would be a classic case of standards-oriented programming. There will be 
many different applications that create files of this type, but client 
applications should not have to worry about where/how the file originated.

Stage 2 (temporal information, meaning): Once SeSVG has been defined, it 
becomes possible to think about including information about the symbols' 
_meanings_. For example, my own software [4] includes MIDI information 
inside chord symbols. Once we know where the chords are in the file, we 
can start adding information to them. I think we can/should leave Stage 
2 for later. SeSVG has many applications, even without the inclusion of 
MIDI information [5]. The rest of this posting is exclusively about SeSVG.
_____________________________
Semantically enhanced SVG (SeSVG):

Note first, that SeSVG is not a replacement for MusicXML.
MusicXML is a de-facto standard for its client applications, and I think 
its unlikely to be replaced. Too much programming effort would be 
involved. I'm very much in favour of Michael's effort to maintain and 
complete it, and would support any effort to turn it into an official 
W3C standard.

MusicXML's clients are score authoring applications, most of which can 
already export SVG.
SeSVG's clients would not only be browsers and SVG editors, but also web 
and desktop applications.
Note however, that SeSVG files can also be created by applications that 
do not import MusicXML. That's important, because W3C standards have to 
be future-proof. We can rely on SVG to represent any graphic in future, 
but we can't assume that MusicXML is going to be able to describe all 
future music notations.
Music notations are restricted only by the authoring application that 
creates them. SeSVG should be both notation agnostic and future-proof. 
Here's how I think that can be achieved:

SVG uses the notion of a _contract_ between the producer and consumer of 
an SVG file when defining namespaces. We can use a similar tactic, to 
define an SVG specialisation by defining a custom data- attribute (lets 
call it data-svgTypeID) in the <svg> element [6]. For example:

<svg xmlns="http://www.w3.org/2000/svg"
     ...
     data-svgTypeID="http://www.../.../seSVG1.html"
     ...
 >
...
</svg>

The data-svgTypeID attribute will be ignored by browsers, but can be 
accessed by both the producer and consumer of the file, who treat it as 
the ID of the SVG file type. It is common (in SVG namespaces) for the ID 
to be a URL pointing to a (machine- or human-readable) description of 
the specialization.

Even if the format description (here in seSVG1.html) is not a formal, 
machine readable schema, the producer of the file can use a standard 
verifier (like the W3C HTML verifier) to confirm that that file being 
produced conforms to its format ID. Such a verifier would be quite 
simple to write, and would be a public part of the standard, used by all 
the producers of such SeSVG files [7]. There might, incidentally, be 
more than one SeSVG type. Hence my use of seSVG1.html above.

In the above example, the consumer of the file (the parsing application) 
only needs to check the value of data-svgTypeID to know how the file has 
been structured by its producer. Its not necessary to follow the link!

The definition (in seSVG1.html above) includes a set of class names that 
can be associated with the basic SVG elements. Some of these are 
containers (<g>, i.e group elements). In addition to the class names, 
the SVG file type also has to define the position in the container 
hierarchy where each class can occur, and how often. The definitions 
will be very much like those of a formal schema, but they actually only 
need to be in informal, human-readable form.

For example, seSVG1.html might define the following container class nesting:

     <g class="system" ...>
         <g class="staff" ...>
             <g class="voice" ...>
                 <g class="chord" ...>
                     ...
                 </g>
                 ...
             </g>
             ...
         </g>
         ...
    </g>

The definition would also provide a list of other elements or containers 
together with their standardized class names and available positions in 
the container hierarchy. For example, a <line class="slur" ... /> would 
probably only be allowed to occur inside a <g class="voice"...>...</g> 
container.

I imagine that the class names and nesting could be related closely to 
MusicXML definitions and/or to the internal class definitions used by 
music notation authoring applications. Note, however, that SVG files 
contain instantiated graphical information, so there shouldn't have to 
be as many definitions as there are in MusicXML. For example, there 
should be less need to describe notehead types since particular 
noteheads have already been instantiated by the authoring software. 
Editors can see what they are editing.

My feelng is that the number of such SeSVG definitions should be kept to 
a minimum so that consumer applications don't have to be too fussy about 
the formats they support.
Note that the above container hierarchy definition says nothing about 
the position of the symbols on the page, or the direction in which they 
are to be read. The above container hierarchy can just as well apply to 
Asian notations that read vertically.

Comments and suggestions would be very welcome.

Have fun! :-)

All the best,
James


[1] For the minutes of our Frankfurt meeting last April see
https://www.w3.org/community/music-notation/
[2] This problem was especially controversial in my correspondence with 
the MEI last May. See the thread at:
https://lists.uni-paderborn.de/pipermail/mei-l/2016/001798.html
If nobody beats me to it, I will be re-opening that debate on the MEI 
list in a few days time. That seems legitimate, since they are a very 
different audience, with different interests.
[3] This insight came during a private email exchange with Daniel 
Spreadbury. Thanks, Daniel, for your patience!
[4] https://github.com/notator/Moritz
https://github.com/notator/assistant-performer
http://james-ingram-act-two.de/open-source/assistantPerformer/assistantPerformer.html
[5] SeSVG can be read by browsers, and includes the SVG metadata 
capabilities, so I can imagine it being used for archiving. Also, there 
are uses in the academic world: if particular places in the score are 
both localised and easy to locate in the file, it becomes feasible to 
write simple applications for adding annotations and/or extracting music 
examples. These use cases would, of course, be enhanced if/when the 
format includes MIDI info.
[6] Thanks to Nikos Andronikos, who suggested the use of a custom data- 
attribute in the following posting:
https://lists.w3.org/Archives/Public/www-svg/2016Nov/0008.html
[7] As far as I know, it is not possible to write a machine-readable 
schema that can be used to describe a specialised SVG type and verify it 
automatically. Maybe I'm wrong. If this proposal is adopted, we should 
consult some experts in advanced SVG to make sure we are not missing any 
tricks.