Re: A working MusicXML engraver?

I'm going to speak to what I imagine our needs may be down the road a ways
-- I understand that this thread is also sharing some information about
various renderers of today, and that's also very useful.

It's very hard to create a reasonable quality notation renderer or editor
without a full test suite. Noteflight, like other products/projects in this
area, has an extensive set of visual engraving regression tests. However, I
am not sure an screenshot-diff approach like Noteflight's or Soundslice's
is going to address the nature of the tests needed to check correctness in
a future MusicXML specification. Consider what we will want to assert. We
are likely to be making assertions about positional relationships between
glyphs based on their registration points, with considerable flexibility as
to other details of layout that are not prescribed by the future spec (and
which are irrelevant to the test results).

To put this another way: I suspect a MusicXML test suite should not test
the details of fully realized, high-quality music engraving. It is probably
going to be a test of whatever MusicXML requires that a renderer do, and no
more:  the small subset of accepted engraving rules and practices that can
be safely prescribed by a future spec. Unless we want an endless and
ultimately unanswerable series of debates on said rules and practices...

...Joe


.            .       .    .  . ...Joe

*Joe Berkovitz*
President

*Noteflight LLC*
49R Day Street / Somerville, MA 02144 / USA
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"

On Tue, Dec 1, 2015 at 1:32 PM, Joshan Mahmud <joshan.mahmud@gmail.com>
wrote:

> That would be very useful Adrian.  VexFlow similarly has a unit test suite
> which tests all functionality of it's codebase and essentially has a single
> page where a set of small unit-based bits of engraving are demonstrated.
> In my project I am essentially doing the same, but instead the 'Arrange'
> part of my test starts with some MusicXML, and the outcome / 'Assert' is
> the rendered music.  It would be nice to have this sort of set up for each
> part of the specification as it shows a practical example which can be run
> (and debugged).  Having this along with screenshots of the original would
> be really really useful!  (Again this is what would be useful to me, and
> not sure if it would suit everyone).
>
> Thanks
> Josh
>
> On Tue, Dec 1, 2015 at 5:17 PM, Adrian Holovaty <adrian@holovaty.com>
> wrote:
>
>> On Tue, Dec 1, 2015 at 1:55 PM, Andreas Wenger <
>> andi.xenoage@googlemail.com> wrote:
>>
>>> But a test suite in this style (MusicXML input and expected result
>>> image) seems to be perfect for me to measure quality. What tests/benchmarks
>>> do the other MusicXML engravers use?
>>>
>>
>> For Soundslice, we have an extensive internal test suite that tests for
>> regressions each time we improve the engraving engine. It takes screenshots
>> and compares them to the baseline (correct) versions. I wrote about the
>> infrastructure here:
>> http://www.holovaty.com/writing/automated-screenshot-tests/
>>
>> I would be happy to make a freely available repository of MusicXML tests,
>> rendered with our engine. Sure, we're proprietary, but our rendering is
>> pretty solid and the only proprietary bit would be the rendering engine,
>> anyway -- not the screenshots or MusicXML files themselves.
>>
>> Adrian
>>
>
>