Audio (Re: CSS2.1 'content', 404 Not Found, and audio files)

csant writes:
> 
> On Wed, 4 Aug 2004 18:17:07 +0200, Chris Lilley <chris@w3.org> wrote:
> 
> > On Wednesday, August 4, 2004, 5:22:25 PM, csant wrote:
> >
> > c> On Wed, 4 Aug 2004 16:34:04 +0200, Chris Lilley <chris@w3.org> wrote:
> >
> >>> This is in contrast to the aural media, which is serialized by the
> >>> speech synthesizer; contrast
> >>>
> >>> h2:before { content: url(welcome-fanfare.mid); }
> >>>
> >>> h2 {cue-before: url(welcome-fanfare.mid); }
> >>>
> >>> In the latter case the fanfare willsound justbefore each h2 is read  
> >>> out.
> >
> > c> Specifying 'content' :before h2 still is serialized by the speech
> > c> synthesizer - it plays the audio file before reading the heading,  
> > exactly
> > c> as the 'cue-before' example does: the latter method additonally  
> > allows you
> > c> to add some aural "padding" around the header by means of the 'pause'
> > c> property on h2, which comes before the element's contents but after  
> > the
> > c> cue.
> >
> > I was contrasting the :before in a visual medium with the cue-before in
> > an aural medium.
> 
> It sounds a bit weired to talk about visual medium when we are dealing  
> with audio files, doesn't it? :)

At first sight (!) yes, but as you noticed, the 'screen' media type
accepts audio properties, since many browsers that use this medium are
quite capable of playing sounds. There aren't any such properties yet,
but it's a longstanding plan to develop them...

I expect 'content: url(welcome-fanfare.mid)' to show a MIDI plug-in,
that has at least a "play" button, so you can listen to the music.
This is the same as when the URL appears in IMG or OBJECT.

If the download fails (not available, wrong format), I would expect to
see a broken image icon.

On the other hand, if this URL is the value of 'background', I expect
it to be ignored, since displaying the plug-in in the background
doesn't make much sense. You won't be able to click the "play" button
anyway.

> 
> The :before example is actually available for all media types, the  
> 'cue-before' example, on the other hand, can only apply to 'aural' (or  
> 'speech'). Synchronizing sounds with the visual display is not matter of  
> CSS, and yet it becomes an issue when you deal with audio files: how is a  
> UA to treat more than one audio file per document?
> 
> h1:before { content: opening_fanfare1.wav; }
> h1:after { content: closing_fanfare1.wav; }
> h2:before { content: opening_fanfare2.wav; }
> 
> Leaving beside the fact that I might here be abusing the 'content'  
> property - if the UA supports 'speech' it might probably "render" the  
> sounds serialized, if it does not support speech and nevertheless supports  
> audio files, this might end up in a very annoying sound-salad! Or are  
> audio files to be excluded from 'content' when there is no support for  
> media 'aural'/'speech'?
> 
> The media groups section <URL:  
> http://www.w3.org/TR/CSS21/media.html#media-groups> seems to specify that  
> 'screen' is comprised in the 'visual' and 'audio' groups.

Correct. And I think that we can leave it to the style sheet author to
make "sound-salad" or something useful, as long as we provide some set
of properties that allows both.

My idea is that an element's background sound is played as soon as the
element becomes visible in the window and stops when the element is no
longer visible. Sounds can be specified to play once or loop. You can
set their volume and you can also specify what happens to sounds of
other elements: interrupt while this element plays or play at a
certain percentage of their volume. Maybe:

    foo {
      sound-appear: url(...);    /* Play when element appears in window */
      sound-visible: url(...);   /* Loops while elt is visible */
      sound-disappear: url(...); /* Play when element disappears */
      sound-hover-in: url(...);  /* Play when mouse enters element */
      sound-hover-out: url(...); /* Play when mouse leaves element */
      sound-focus-in: url(...);  /* Play when elt gets focus */
      sound-focus-out: url(...)  /* Play when elt loses focus */
    }
    foo:hover, foo:hover {
      sound-visible: url(...)    /* Different sound that loops */
    }

With some control over mixing of sounds, that might become:

    foo { sound-hover-in: url(bang.ogg) 80%/mute }

meaning that the sound plays at 80% of its intrinsic volume and all
other sounds are muted while it plays. If several elements are visible
(which is normally the case, of course), we need a rule to determine
what is their "z-order": e.g., a child can mute its parent, but not
the other way round. Or maybe we just need to define the order in
which sounds start and then the last to start can muffle all others,
which would allow the parent's hover sound to interrupt the child's
backgound sound.

Maybe each sound should be a comma-separated list of alternative URLs,
for the case that the first sound is not available or in the wrong
format. Maybe there should be some keywords for generic sounds at the
end of the list: 'beep', 'ting', 'white-noise', 'boom'...



Bert
-- 
  Bert Bos                                ( W 3 C ) http://www.w3.org/
  http://www.w3.org/people/bos/                              W3C/ERCIM
  bert@w3.org                             2004 Rt des Lucioles / BP 93
  +33 (0)4 92 38 76 92            06902 Sophia Antipolis Cedex, France

Received on Thursday, 5 August 2004 17:43:37 UTC