Formal summary of music domain literature

To the RQTF

I've completed my more formal summary of the literature associated with the domain review related to music accessibility. There is much more detail n the respective papers but hopefully it goes some way to answering Janina's questions below in terms of the logic behind the research approaches. 

Could I please trouble someone (Dave?) to add this summary to the music domain literature wiki? That way the literature I've cited can easily be matched up the full citation. 

Thanks everyone and summary follows. 

Scott. 

-----


INTRODUCTION

The literature regarding the accessibility in the music domain tends to be weighted towards print disability groups due to the challenges in reading musical notation. Within this group research has tended to focus on addressing the issues for two distinct groups: people with a vision-related disability and people with dyslexia. 

VISION 

Historically, much of the effort in making music accessible for people with vision-related disabilities focused on connecting music software with screen readers. In a recent study based on a large research project known as Visually impaired musicians’ lives (VIML), (Baker, 2016) the views of blind and visually impaired amateur and professional music teachers, along with music teachers that worked with visually impaired students, were asked about the core issues in using popular technologies to gain access to musical notation.  The response was that the issue remains a challenge despite some promising past endeavours.  

One example of a solution from the 1990s was created by producers of electronic music whereby programmers created software plug-in solutions for inaccessible music software (for notation and recording). This included the development of a bridge between the JAWS screen reader and popular notation software. However, the rate of software development in both the music notation software and assistive technology software has meant that solutions have a short shelf-life and few programs today have accessible solutions available. 

With this in mind, there are currently two schools of thought relating to accessible music notation for people with a vision disability: the first is to explore new ways of delivering musical notation such as the use of audio or haptic feedback, while the second is to focus on making Braille music accessible in a digital format.  

AUDIO AND HAPTIC SOLUTIONS 

One popular research approach is the use of image processing to convert musical notation into audio in real-time which is then conveyed to the musician. One approach (Genfang, Wenjun, and Qiuqiu, 2009) focused on analysing and structuring elements of musical notation and the basic algorithms of mathematical morphology. This approach essentially converts the musical notation into MIDI with the output of the musical score being recognised over 94% of the time indicating a highly accurate image to audio conversion process. The musician can then use the audio to understand the notes and tempo that need to be played. 

Another approach to providing accessible music is described as ‘Spoken Music’. This is when music is presented as a series of played music fragments, each a few bars long, followed by a detailed spoken description of all the items in the music score (Crombie, et. al., 2002). The paper argues  against the use of Braille music due to its lack of familiarity with younger people, the delays in getting it produced and lack of broad awareness, instead focusing on the delivery of audio fragments through the use of assistive technology plug-ins for popular music editing software such as Finale 6, with the ability to print as Braille music should a physical document still be required. The research suggests this is the best of both worlds as its primary focus is on the delivery of an audio-based musical score without the need to learn Braille music, but it is still possible to return to an older Braille format if necessary. 

Another research project that was undertaken due to the challenges in creating Braille focused on the conversion of Eastern music notation into an audio-based interface (Kiriella, et. al.,2014). The research was sparked from a perception that Braille music was impractical and as such focused on trialling audio and haptic delivery mechanisms. 

BRAILLE MUSIC 

In relation to the second research aspect focusing on Braille music, a research study in Brazil developed software to teach primary school students Braille music (Borges and Tomé, 2014). The idea of the software was for both sighted and blind students to participate in a shared music space leading to pedagogical advantages and stronger learning outcomes. 

Two other studies focused on the use of new mark-up techniques to improve the digital access of Braille music. One example (Brown and Robinson 2004) focused on the creation of 4DHML to mark-up real-time modelling when content was inaccessible, while another research project (Encelle, et. al, 2009) proposed the creation of a Braille Music Mark-up Language (BMML). The concept of BMML is as a result of XML being widely used for musical applications, and the lack of ability for blind musicians to share their musical scores. In this instance, BMML would address this issue moving Braille music into a modern web context. A further research study (Encelle, et. al., 2006) has also focused on the use of XML to convert musical notation into Braille music to make the notation more accessible and easily shared online. 

DYSLEXIA 

The second print disability group that features in the music accessibility domain literature relates to dyslexia. Many of the characteristics of dyslexia—such as difficulties with decoding written symbols, phonemic awareness, physical coordination, and readable handwriting—may adversely affect music learning (Nelson and Hourigan, 2016). One research study focused on examining the perceptions of five professional musicians with dyslexia as they reflected on their experiences learning music. Recommendations from the study included support for multisensory teaching, isolating musical components, learning of jazz and popular music, using technology, and small group instruction (Nelson and Hourigan, 2016). 

Another study focused on the use of an iPad tablet computer and a musical notation app to help students with Dyslexia as a multi-sensory instructional tool (Witmer, 2015). The study concluded that the use of the tablet for guided practice in conjunction with instruction was significantly more effective at increasing the ability of students to recognize musical staff notation than traditional learning approaches. 


ISSUES ACROSS PRINT DISABILITY GROUPS

While the approaches to making music accessible vary in approach, there are some common issues shared between disability groups. The two recurring themes are that the technology moves faster than solutions can be found to support it, and that education professionals lack the training and awareness to provide students with print disabilities the solutions they need.  (Baker, 2016) (Knight and Lagasse, 2012)). This has been particularly highlighted when music teachers themselves have a print disability as there is a perceived inability to employ such teachers regardless of their musical and teaching ability due to an inability for the educational institution to support their delivery of teaching to students. (Parker and Draves, 2017).

CONSIDERATIONS FOR SOLUTIONS IN DOMAIN-SPECIFIC AREAS

While there are a number of different approaches in terms of the technologies used to address accessibility issues in the music domain, there are a broad set of considerations raised in one research study that are applicable to both this domain and other domain-related accessibility pursuits. 

The list of considerations (Power and Jürgensen, 2018) have been identified as follows: 
 
• Interface requirements should be generalised and not dependent on specific applications so that identified benefits can be used broadly; 
• Multi-modal interfaces must be brought to mainstream applications; 
• Awareness of universal access must be increased; and
• Involvement of the target user group must be included at all levels of design. 

---

End of summary. 

Dr Scott Hollier 
Digital Access Specialist 
Mobile: +61 (0)430 351 909
Web: www.hollier.info
 
Technology for everyone
 
Looking to upskill your staff with digital access training? Fill the room for one flat fee. 
 
Keep up with digital access news by following @scotthollier on Twitter and subscribing to Scott’s newsletter. 

-----Original Message-----
From: Janina Sajka <janina@rednote.net> 
Sent: Wednesday, 7 November 2018 9:56 PM
To: Scott Hollier <scott@hollier.info>
Cc: Research Questions Task Force <public-rqtf@w3.org>
Subject: Re: Music domain literature summary

Thanks for the great summary Scott.

Let me ask the threshold questions in my mind:

1.) Does anyone anywhere in the literture point out the most obvious
problem with braille music? Namely that the only musician who can concievably read braille while practicing or performing music is a singer? Because every other musician must use their hands in music-making?

2.) Has anyone investigated using technology to enhance and and
train the most important ability every musician needs to acquire? A capable and accurate ear?

I'm frankly frustrating to hear of people talking about somehow reproducing the written presentation of music where the consumption of that modality is so obviously unavailable, while at the same time ignoring the most obvious and important route.

Having said this, I should add I'm not surprised. Western music from the common practice era has become overly invested in the written score to the extent that most classical musicians are incapable of playing without a piece of paper on the music rack in front of their face. It's frankly a scandalous situation from the socio-historical point of view, and one that leaves pop and jazz musicians shaking their heads in bewilderment.

<sigh>

Janina

Scott Hollier writes:
> To the RQTF
> 
> As promised, here is a summary of the music domain literature. The summary is based on the literature references that I collated and Dave put on the music domain wiki.
> 
> To summarise:
> 
> 
>   *   The literature is heavily focused on the accessibility of music as it relates to people with print disabilities, particularly dyslexia and vision
>   *   Dyslexia-related research tends to focus on the use of multisensory approaches combining visual and audio feedback through commercial products such as the use of an iPad with specialist apps that guide learners through music notation. This appears to have some degree of success.
>   *   For vision, the research is divided between two schools of thought:
>      *   How best to make Braille music accessible. The argument here is that Braille music works well but needs to be converted into an electronic mechanism
>      *   How best to move away from Braille music entirely and find a better solution  the argument here is that Braille music isn’t known by young people, takes time to convert and is expensive
>   *   In terms of converting Braille, research tends to focus more on the conversion mechanisms such as a Braille Music Mark-up Language and largely XML-based solutions.
>   *   In the case of moving away from Braille entirely, a combination of haptic and audio-based solutions whereby music is scanned and reproduced in small pieces of audio or haptic feedback to deliver the information as required in the score
> 
> In addition, there’s one paper which in my opinion Is a must-read for all of us and goes to the bigger picture of domain-specific guidance. The paper is:
> 
> Power, Christopher and Jürgensen, Helmut (2010) Accessible 
> presentation of information for people with visual disabilities
> Journal: International Journal Volume: 9 Issue: 2 Pages: 97-119
> 
> Below I’ve quoted the core summary bullet points from the paper but it reads a bit bumpy, I’m guessing due to the OCR software used to get it in the journal. The message is really good though.
> 
> “• Interface requirements need to be abstracted away from specific applications. Specific applications provide a means of testing the effectiveness of interface theories and designs. These applications have very specific human and technological factors which make them successful in achieving their goals. These factors need to be generalized in such a way that future research and commercial systems can include them in new applications. • Multimodal interfaces must continue to be brought to mainstream applications. While any type of provided feedback is of benefit to a user, material translated for the people with visual disabilities should use both audio and tactile output. While audio output is certainly easier to manufacture, it is too serial to communicate all information effectively. It is clear from previous examples of technology, such as the Optacon and the IVEO tablet, that devices and applications which include tactile feedback are more readily accepted by the user community. • Further work on automatic transcoding is required. There are several examples of transcoding for each type of media discussed in this paper. Transcoding, if it can be accomplished without the aid of a human assistant, provide independence for the user with a visual disability in controlling their access to information. Also, it is essential that the process be examined from the view of the document as a whole, so that one tool can render all information contained in a single document. • Further automatic and semi-automatic testing tools need to be developed. While there are many tools that provide automatic testing for accessibility on the web and in other domains, these tools address a small subset of accessibility problems. • Awareness of universal access must be increased. Tools for transcoding and verification of material will continue to be ineffective if those who are in greatest need of them are unaware of their existence. Media transcribers, developers and students all must be informed of the challenges which exist for those with disabilities, so that they can look for specific tools.
> • Involvement of the target user group must be sought at all levels of design, implementation and testing. There is a need to include the target user group at all levels of the research process. There are several examples in the literature where tools have been designed without the input of the users, and then tested without participation of that community. However, certain techniques for acquiring testing data produce accessibility concerns of their own. An example of this is the use of time diaries for people with visual disabilities, which were shown to have their own set of unique problems [4]. It is important for researchers to be aware of such problems, so that development and testing plans can be adjusted appropriately.”
> 
> Scott
> 
> [Scott Hollier logo]Dr Scott Hollier
> Digital Access Specialist
> Mobile: +61 (0)430 351 909
> Web: www.hollier.info<http://www.hollier.info>
> 
> Technology for everyone
> 
> Looking to upskill your staff with digital access training<http://www.hollier.info/consultancy/>? Fill the room for one flat fee.
> 
> Keep up with digital access news by following @scotthollier on Twitter<https://twitter.com/scotthollier> and subscribing to Scott’s newsletter<mailto:newsletter@hollier.info?subject=subscribe>.
> 



-- 

Janina Sajka

Linux Foundation Fellow
Executive Chair, Accessibility Workgroup: http://a11y.org


The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI)
Chair, Accessible Platform Architectures http://www.w3.org/wai/apa

Received on Friday, 16 November 2018 12:54:29 UTC