W3C home > Mailing lists > Public > w3c-wai-ig@w3.org > April to June 1999

Re: wave file as alt tag?

From: Peter Meijer <meijer@natlab.research.philips.com>
Date: Mon, 17 May 1999 20:54:47 +0200
Message-Id: <374065F7.24A05FDB@natlab.research.philips.com>
To: w3c-wai-ig@w3.org

Al Gilman wrote

> Yes, technically, the sound that is attached to the flag icon could be the
> sonification of the image.  But this would be a triumph of technology over
> communication.
> In popular culture this image already has an associated sound.  The sound
> for the image of a waving stars and stripes is that of Arthur Fiedler
> conducting the Boston Pops in Sousa's "Stars and Stripes Forever" on the
> Esplanade on the Fourth of July.
> For inclusion of the reading-impaired, it is important to honor the
> associations of sight, sound and text that already exist as cliches in the
> lexicon of popular culture.

and Al also wrote in reply to David Poehlman

> Of course you are right.  The expanding possibilities are more important
> that one arguable example.  I was impressed with the ability of first-time
> users to grasp simple shapes from their vOICe sonification when we played
> with this at CSUN.
> I wonder if there aren't also exciting, world-expanding possibilities to be
> explored for communication between people with different disabilities if a
> lexicographer were to catalog the multimedia cliches that are widely
> recognized today and hence already work as code because the cross-medium
> associations are present and consistent for many people.

Thank you, Al. I agree that using existing associations is a good
and attractive idea. I suppose that perhaps most US citizens would 
indeed have the association of the US flag with the particular piece
of music that you mentioned, but in the rest of the world this may 
already be different. Still, maybe my example was not a very good 
one to make a point about generality. I sincerely sympathize with 
using intuitive associations wherever possible, but I think it will
be very hard to always find associations that are culturally neutral
and unambiguous. Associations are known to work quite well with 
limited image sets, such as the relatively small number of windows
controls and event types in a computer GUI, but for arbitrary images
I expect the scheme to break down for lack of suitable "multimedia 
cliches" in the envisioned lexicographer's catalog.

This is why the attempt arose to define a completely general and
unambiguous image sonification, being semantically neutral and
amenable to automatic processing. Still, I admit it has its own 
share of disadvantages, particularly the interpretation burden on
the user. Learning a new language is never easy, so it had better
be a really powerful language to justify all the effort involved.
Lots of open issues in this area.

Partly just for fun, but the US flag image turned 90 degrees to 
the right sounds like (download the 88K .wav or the 16K MP3)


The stripes that now run vertically sound as noise bursts, and the
stars are now in the upper right (high-pitched on the right). With
another 90 degree rotation to the right, the stars would end up in
the lower right (low-pitched on the right), and the stripes would
sound like tone bands again, because they run horizontally again. 
I wonder what the lexicographer would find, or what Sousa's "Stars
and Stripes Forever" would sound like when turned 90 degrees to the
right? And when turned another 90 degrees?

Best wishes,

Peter Meijer

Soundscapes from The vOICe - Seeing with your Ears!
Received on Monday, 17 May 1999 14:44:25 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 13 October 2015 16:21:04 UTC