Practical lessons from HTML Audio controls

This is a note on <audio controls>, a modern UI widget introduced in 
HTML5 and its implementation in IE and WebKit.

Other widgets, such as <select> were introduced long ago. Through 
improved CSS support, we're able to style old element appearances though 
that styling may not carry forward to the element when it is active.

With <select> we can do something like: <select 
style="-webkit-appearance: none;" />. This "hides" the toggle arrow 
typically seen in select and lets us do more customization of the select 
presentation. We're still not able to style the <option> menu of 
<select> beyond the basics of background and text.

So, let's consider <select> as the old way.

The new way can be seen with <audio controls>.

1. Exposing the full API.

The actions of the <audio> element are pretty well exposed to the 
scripting environment. This allows authors to create their own 
"controls" implementation and simply use the <audio> tag. We can do 
something similar with <select> by hiding the tag and using stock HTML 
and ARIA, of course. That's a big part of why ARIA came about.

2. Exposing sub-element styling

The components of the <audio> element can be styled using 
pseudo-selectors in WebKit.

They are lengthy, but each UI element inside of the audio tag can be 
changed, example:
audio::-webkit-media-controls-volume-slider-container {
  background: blue;
}

3. Allowing partial implementations.

This is one is most similar to traditional input elements. An <audio 
controls> item may simply spawn a new window or controller which blocks 
the visible UI. That's one of the benefits of native tags, they can 
degrade gracefully or otherwise be enhanced through user preferences.

As for best practices that come from this, I've learned a few things:

1. Pseudo-selectors are difficult to predict.

The Web Components project may help standardize this process. There is 
no standard saying that the <audio> element should have a volume slider, 
or that the slider should be in a particular position in the element. 
Still, what we have with WebKit lets us do a lot.

We can transform widgets, change colors, we can even draw over them with 
<canvas>.

2. Keyboard events are important and somehow still get neglected.

Until recently, the <audio> tag didn't receive focus in WebKit.
One would have to do something like this:
<button onclick="audio.pause()" tabindex="-1"><audio tabindex="0" 
controls /></button>.

The audio tag still does not respond well in WebKit to keyboard events. 
For a good contrast, take a look at IE. In IE9, one could focus, and use 
arrow keys to skip ahead as well as change the volume.

Microsoft's keyboard events seem to work quite well. In some sense, they 
set a standard for keyboard events. Left and right scan through the 
audio file in increments, up and down change volume, space and enter pause.

3. Element appearance should be flexible to height width and scale.

WebKit's pseudo-selectors work pretty well. We can't change the position 
of elements but we can hide them with display none and we can draw over 
them with Canvas. We know their height in proportion to the host 
element. We know width scales with the timeline filling the extra room 
and we know that the play button and the volume buttons span the 
timeline area.

We can estimate that WebKit will, at some point, allow <audio controls> 
with very narrow widths. They are currently allowed but result in play 
and volume buttons overlapping each other.
It seems reasonable that the play button would take priority. Currently, 
authors have to handle the situation manually through the use of 
pseudo-selectors.

We've already seen with <input type="checkbox" /> that implementations 
can be very uneven when it comes to behaving well with CSS height and 
width. It's fortunate that we have some tools at our disposal to work 
around these issues in some native implementations.
One final note on scaling: bitmaps matter.

Both Chromium and IE teams have noticed that their play and volume 
buttons become pixelated when the element is enlarged. That's because 
they're using simple bitmaps and they assumed smaller sizes.
The first response to fix this was to create larger bitmaps. This allows 
the implementation to adapt, showing a higher resolution image when the 
element is larger.

It's likely that vector graphics will also be used in future releases. 
So we'll see bitmaps (PNG) in use when the size of the element is small, 
say around 50 pixels in height, and we'll see vector (SVG) when the 
elements are larger, maintaining quality.

In the meantime, with WebKit, authors can apply fixes , if they're up 
for the work.

-Charles

Received on Wednesday, 25 April 2012 20:58:40 UTC