Re: Semantically enhanced SVG (proposal)

I think there are several issues with this proposal which I will outline
here:

Name: The name "Semantically enhanced SVG (SeSVG)" is too far reaching and
ambiguous. It does not identify that the semantics are limited to musical
semantics. There are other graphics that can also contain semantic info
that are not music notation, so I would propose a different name that
limits the scope of this semantic to musical notation elements. For
example, a name such as "Musically Enhanced SVG" would better because it
identifies this format to musically semantic elements only.

The need for yet another standard: I am extremely wary of the need for yet
another standard. Does this actually solve a problem for the entire W3C CG,
or just for a few developers? I am not yet convinced that this standard is
needed; it appears to be a very limited use case. This is not say that it
is inherently unnecessary (ex: Verovio uses a similar approach), but I do
have concerns that this falls within the scope of the activities of the
group (or where within the scope of the group's activities this is best
suited).

Stage 1: This stage seems reasonable. I will call this the microformat
approach. This biggest impediment though is the reliance on having the
standard agreed upon before any client can develop against it. This won't
happen because either 1) clients will start implementing the standard
before it is agreed upon and/or 2) clients will not implement the standard
until it is fully formed, which could lead to a slow adoption rate.

Stage 2: This stage seems to be the most problematic. If I understand this
correctly, Mr. Ingram is suggesting embedding MIDI data inside of SVG
elements. I would strongly advise against this approach. Instead, I would
recommend creating a relation between the SVG data and the MIDI data. This
would enhance interop with SVG clients that don't understand MIDI, as well
as with MIDI clients that don't understand SVG.

Reliance on SVG: To quote Mr. Ingram "...W3C standards have to be
future-proof. We can rely on SVG to represent any graphic in the future, be
we can't assume that MusicXML is going to be able to describe all future
music notations". In regards to MusicXML, Mr. Ingram is correct that we
cannot assume that MusicXML will represent all music notations, but I feel
very strongly that one of the primary purposes of this W3C CG is to improve
and evolve MusicXML (and/or other/future notation standards such as MEI or
MNX). To say that we cannot describe future music notations appears to
indicate Mr. Ingram's lack of faith in the CG to develop music notations in
the future. Whether MusicXML, MEI, MNX, or some other notation format
becomes a W3C or de facto standard, I believe we can continue to evolve
that/those standard(s) to meet the needs of the CG group.

Support for multiple standards: Mr. Ingram only suggests MusicXML -> SVG.
Mr. Hankinson and Mr. Pugin demonstrated that MEI -> SVG would also be
possible, as Mr. Ingram's proposal very closely aligns with Verovio's
implementation. I also wonder if it would be possible to preserve enough
semantics to convert a semantically enhanced SVG back to MusicXML or MEI. I
would be very much in favor of a bi-directional approach to enhance interop.

These are just my initial concerns. While I think that some form of
semantically enhanced SVG or an SVG with a microformat would be beneficial
to some in the CG, I do not feel that now is the appropriate time work on
this. Perhaps this should be the work of a subcommittee rather than the
work of the entire group?

J. Sawruk

On Mon, Nov 28, 2016 at 3:43 AM, Laurent Pugin <lxpugin@gmail.com> wrote:

> The structure of the SVG generated by Verovio is indeed exactly the one
> you are suggesting (with the only difference that a "voice" is a "layer" in
> MEI)
>
> <g class="system">
>     <g class="staff">
>         <g class="layer">
>             <g class="chord">
>                 <g class="note">
>                 </g>
>             </g>
>         </g>
>     </g>
> </g>
>
> You can find more information about it in this paper
> http://www.terasoft.com.tw/conf/ismir2014/proceedings/T020_221_Paper.pdf
>
> Best wishes,
> Laurent
>
>
> On Fri, Nov 25, 2016 at 3:16 PM, Andrew Hankinson <
> andrew.hankinson@gmail.com> wrote:
>
>> It should be noted that this is pretty much exactly how Verovio renders
>> SVG with MEI.
>>
>> See "SVG Structure" at http://www.verovio.org/structure.xhtml
>>
>> We use SVG versions of the SMuFL fonts for the graphical primitives.
>>
>> -Andrew
>>
>> On Nov 25, 2016, at 10:01 AM, James Ingram <j.ingram@netcologne.de>
>> wrote:
>>
>> Hi,
>>
>> I'd like to propose that the W3C Music Notation Community Group develop a
>> semantically enhanced SVG file type for music scores.
>> _____________________________
>> Summary:
>>
>> Semantically enhanced SVG (SeSVG) is a specialized SVG file type
>> containing identifiable music notation objects. All such objects are
>> defined using "class" attributes on the basic SVG elements. SeSVG files are
>> just ordinary SVG files structured in a particular way, so it should be
>> easy for applications that already export SVG to export SeSVG.
>> Interesting client applications can written because the contained objects
>> are localized and identifiable, and the files can be reliably parsed.
>> Client applications will not only be browsers and SVG editors, but also
>> web and desktop applications.
>> SeSVG is not competing with MusicXML (whose clients are music notation
>> authoring tools).
>> The posting ends by describing how SeSVG files can be defined and managed.
>> _____________________________
>> Background:
>>
>> Following the Community Group meeting in Frankfurt last April [1] and the
>> subsequent email exchanges, we decided that everyone should go off and work
>> on their own for a while. I've now made some progress that I'd like to
>> share.
>>
>> The main problem, for me, in previous discussions, was understanding what
>> everyone meant by "semantic". For me, "semantic" refers both to the naming
>> of symbols (in files and elsewhere) and to their meanings. Many problems
>> disappear if these two concepts are kept strictly separate [2].
>>
>> Another problem was that I have been developing a hybrid SVG format
>> containing both ordinary SVG and MIDI information. In trying to explain
>> that format, I was trying to tackle too many problems at once.
>> I now think that it would be a good idea to step back a bit, and approach
>> the problem into two stages [3].
>> Interestingly, these two stages exactly reflect the distinction between
>> symbol names and their meanings.
>>
>> Stage 1, SeSVG (graphics, names): We should first define a standard way
>> to write music graphics, independently of the meanings of the symbols.
>> Since this will be a purely graphic standard, it can be expressed as an SVG
>> specialisation. The standard should include both standard names for the
>> symbol classes (e.g. "staff", "slur" etc.) and a description of how they
>> nest. For more details of how this can be made to work, and work flexibly,
>> see below.
>> Note that most music score authoring applications already export ordinary
>> SVG (all of them can, in principle), so it should be fairly straightforward
>> for them to adapt their code to export SeSVG. But note also, that the SeSVG
>> standard has to be agreed before anyone can actually write an
>> implementation that creates an instantiation. This would be a classic case
>> of standards-oriented programming. There will be many different
>> applications that create files of this type, but client applications should
>> not have to worry about where/how the file originated.
>>
>> Stage 2 (temporal information, meaning): Once SeSVG has been defined, it
>> becomes possible to think about including information about the symbols'
>> _meanings_. For example, my own software [4] includes MIDI information
>> inside chord symbols. Once we know where the chords are in the file, we can
>> start adding information to them. I think we can/should leave Stage 2 for
>> later. SeSVG has many applications, even without the inclusion of MIDI
>> information [5]. The rest of this posting is exclusively about SeSVG.
>> _____________________________
>> Semantically enhanced SVG (SeSVG):
>>
>> Note first, that SeSVG is not a replacement for MusicXML.
>> MusicXML is a de-facto standard for its client applications, and I think
>> its unlikely to be replaced. Too much programming effort would be involved.
>> I'm very much in favour of Michael's effort to maintain and complete it,
>> and would support any effort to turn it into an official W3C standard.
>>
>> MusicXML's clients are score authoring applications, most of which can
>> already export SVG.
>> SeSVG's clients would not only be browsers and SVG editors, but also web
>> and desktop applications.
>> Note however, that SeSVG files can also be created by applications that
>> do not import MusicXML. That's important, because W3C standards have to be
>> future-proof. We can rely on SVG to represent any graphic in future, but we
>> can't assume that MusicXML is going to be able to describe all future music
>> notations.
>> Music notations are restricted only by the authoring application that
>> creates them. SeSVG should be both notation agnostic and future-proof.
>> Here's how I think that can be achieved:
>>
>> SVG uses the notion of a _contract_ between the producer and consumer of
>> an SVG file when defining namespaces. We can use a similar tactic, to
>> define an SVG specialisation by defining a custom data- attribute (lets
>> call it data-svgTypeID) in the <svg> element [6]. For example:
>>
>> <svg xmlns="http://www.w3.org/2000/svg" <http://www.w3.org/2000/svg>
>>     ...
>>     data-svgTypeID="http://www.../.../seSVG1.html"
>> <http://www.../.../seSVG1.html>
>>     ...
>> >
>> ...
>> </svg>
>>
>> The data-svgTypeID attribute will be ignored by browsers, but can be
>> accessed by both the producer and consumer of the file, who treat it as the
>> ID of the SVG file type. It is common (in SVG namespaces) for the ID to be
>> a URL pointing to a (machine- or human-readable) description of the
>> specialization.
>>
>> Even if the format description (here in seSVG1.html) is not a formal,
>> machine readable schema, the producer of the file can use a standard
>> verifier (like the W3C HTML verifier) to confirm that that file being
>> produced conforms to its format ID. Such a verifier would be quite simple
>> to write, and would be a public part of the standard, used by all the
>> producers of such SeSVG files [7]. There might, incidentally, be more than
>> one SeSVG type. Hence my use of seSVG1.html above.
>>
>> In the above example, the consumer of the file (the parsing application)
>> only needs to check the value of data-svgTypeID to know how the file has
>> been structured by its producer. Its not necessary to follow the link!
>>
>> The definition (in seSVG1.html above) includes a set of class names that
>> can be associated with the basic SVG elements. Some of these are containers
>> (<g>, i.e group elements). In addition to the class names, the SVG file
>> type also has to define the position in the container hierarchy where each
>> class can occur, and how often. The definitions will be very much like
>> those of a formal schema, but they actually only need to be in informal,
>> human-readable form.
>>
>> For example, seSVG1.html might define the following container class
>> nesting:
>>
>>     <g class="system" ...>
>>         <g class="staff" ...>
>>             <g class="voice" ...>
>>                 <g class="chord" ...>
>>                     ...
>>                 </g>
>>                 ...
>>             </g>
>>             ...
>>         </g>
>>         ...
>>    </g>
>>
>> The definition would also provide a list of other elements or containers
>> together with their standardized class names and available positions in the
>> container hierarchy. For example, a <line class="slur" ... /> would
>> probably only be allowed to occur inside a <g class="voice"...>...</g>
>> container.
>>
>> I imagine that the class names and nesting could be related closely to
>> MusicXML definitions and/or to the internal class definitions used by music
>> notation authoring applications. Note, however, that SVG files contain
>> instantiated graphical information, so there shouldn't have to be as many
>> definitions as there are in MusicXML. For example, there should be less
>> need to describe notehead types since particular noteheads have already
>> been instantiated by the authoring software. Editors can see what they are
>> editing.
>>
>> My feelng is that the number of such SeSVG definitions should be kept to
>> a minimum so that consumer applications don't have to be too fussy about
>> the formats they support.
>> Note that the above container hierarchy definition says nothing about the
>> position of the symbols on the page, or the direction in which they are to
>> be read. The above container hierarchy can just as well apply to Asian
>> notations that read vertically.
>>
>> Comments and suggestions would be very welcome.
>>
>> Have fun! :-)
>>
>> All the best,
>> James
>>
>>
>> [1] For the minutes of our Frankfurt meeting last April see
>> https://www.w3.org/community/music-notation/
>> [2] This problem was especially controversial in my correspondence with
>> the MEI last May. See the thread at:
>> https://lists.uni-paderborn.de/pipermail/mei-l/2016/001798.html
>> If nobody beats me to it, I will be re-opening that debate on the MEI
>> list in a few days time. That seems legitimate, since they are a very
>> different audience, with different interests.
>> [3] This insight came during a private email exchange with Daniel
>> Spreadbury. Thanks, Daniel, for your patience!
>> [4] https://github.com/notator/Moritz
>> https://github.com/notator/assistant-performer
>> http://james-ingram-act-two.de/open-source/assistantPerforme
>> r/assistantPerformer.html
>> [5] SeSVG can be read by browsers, and includes the SVG metadata
>> capabilities, so I can imagine it being used for archiving. Also, there are
>> uses in the academic world: if particular places in the score are both
>> localised and easy to locate in the file, it becomes feasible to write
>> simple applications for adding annotations and/or extracting music
>> examples. These use cases would, of course, be enhanced if/when the format
>> includes MIDI info.
>> [6] Thanks to Nikos Andronikos, who suggested the use of a custom data-
>> attribute in the following posting:
>> https://lists.w3.org/Archives/Public/www-svg/2016Nov/0008.html
>> [7] As far as I know, it is not possible to write a machine-readable
>> schema that can be used to describe a specialised SVG type and verify it
>> automatically. Maybe I'm wrong. If this proposal is adopted, we should
>> consult some experts in advanced SVG to make sure we are not missing any
>> tricks.
>>
>>
>>
>>
>