W3C home > Mailing lists > Public > whatwg@whatwg.org > April 2017

Re: [whatwg] <audio> metadata

From: Philip Jägenstedt <philip@foolip.org>
Date: Mon, 10 Apr 2017 04:44:42 +0000
Message-ID: <CAKHWUkb6cWMwkZTGa9NXXZTuDCh9OOiU_ei_iavHud_v8PqfbQ@mail.gmail.com>
To: Andy Valencia <ajv-cautzeamplog@vsta.org>, whatwg@whatwg.org
On Mon, Apr 10, 2017 at 8:38 AM Andy Valencia <ajv-cautzeamplog@vsta.org>
wrote:

> What follows is a first pass at addressing a missing ability
> when dealing with Internet streams, usually radio ones.
> Comments and suggestions are quite welcome; as my first
> attempt--ever--at submitting to this group, apologies if
> I've made any mistakes in how I've proceeded.
>
> Thanks,
> Andy Valencia
> Contact: https://vsta.org/contact/andy
>
> ========================
> Proposal for enhancement to HTML5 <audio> element to better support
> metadata provided by streams.
>
> # Background
>
> Browsers have an <audio> element which can directly play streaming
> URL's.  When the stream is an Internet radio station, the stream
> almost certainly offers metadata; typically, the artist and track,
> and often an album cover URL as well.  While modern browsers can
> play these streams, there is no way to access this metadata.
>
> Media elements in modern browsers _do_ have the notion of metadata,
> but in current standards this is limited to static characteristics
> such as duration.  When listening to a radio stream, the metadata will
> change as successive tracks are played.
>
> # Stream Delivery of Dynamic Metadata
>
> A moderately detailed description of current practices in providing
> metadata is provided at:
>
>     http://www.smackfu.com/stuff/programming/shoutcast.html
>
> (One detail glossed over in this page is that older streaming servers
> start their GET response with "ICY 200 OK" rather than a standard
> HTTP response.  This technically means the response is HTTP 0.9
> without headers; the headers are present, but must be processed by
> the application and trimmed from the start of the media stream
> in the GET response.  Apple Safari already declines to play 0.9
> media elements within a post-0.9 page, and there are signs that
> Chrome will follow suit.  Thus, accomodating this older mode should
> be considered optional.)
>
> Newer streaming server versions use a proper HTTP response, with
> the documented elements in the HTTP response header.
>
> Because the metadata is encoded in a general attribute=value
> format, a wide variety of metadata may be encoded.  By convention,
> at least the attributes "StreamTitle" and "StreamUrl" will be
> included in the response.  Also by convention, StreamTitle is
> structured as "Artist - Track Name".
>
> # API Considerations
>
> While listening to streams with metadata is a compelling application
> of web technology, it is probably a tiny percentage of the use
> of <audio> elements.  Thus, this proposal describes a mechanism
> which leaves the default browser behavior unchanged.  It also
> gracefully degrades when presented to a non-implementing
> browser, still permitting stream play while in the existing
> non-metadata fashion.
>
> This proposal is designed with the current state of streaming,
> currently depending on the HTTP Icy-MetaData header element.  However,
> this specific detail is abstracted, in recognition that streaming
> technology could evolve in the future.  The intention is that this API
> will remain stable even if the details of metadata request and
> delivery change.
>
> As noted, current HTML5 media elements do have some metadata support.
> This API is also structured to permit these existing elements to
> participate in this API if desired.
>
> # <audio> element changes
>
> These enhancements are activated when the <audio> element has the
> new attribute "dynamicmetadata" present.  Without this attribute,
> no metadata request header is added to the stream HTTP request, and no
> other attributes of this proposal will be acted upon even if present.
>
> If the server response indicates support for dynamic metadata,
> on each metadata update, the <audio> element's attributes are changed
> to reflect the latest received metadata.  For each "Attribute=Value"
> in the metadata update, an attribute "metaAttribute" with value "Value"
> will
> be added to the <audio> element.  Previous metadata attributes which
> are not present in the latest update are left untouched.  The browser
> must verify well-formed identifier names for each Attribute, and
> quietly reject ill-formed names.  It can also apply checks on the
> Value.  (Implementors are reminded that the Value, because it encodes
> information such as names, might contain a wide range of character
> encodings.)
>
> The <audio> element adds the hook "onmetadatachange", connecting to
> a function which is called on each update of metadata.  This
> function is called with an event, which includes the attribute
> "changed" with a value which is a list of Attribute names which
> are present in the update.
>
> # Example code
>
> <code>
> function new_meta(ev) {
>     const changes = ev.changed;
>     if (changes.indexOf("StreamTitle") >= 0) {
>         const title = player.metaStreamTitle;
>         const idx = title.indexOf(" - ");
>         if (idx == -1) {
>             // "Just a plain old piece of text"
>             artist.textContent = title;
>             track.textContent = "";
>         } else {
>             // <artist> - <title>
>             artist.textContent = title.slice(0, idx);
>             track.textContent = title.slice(idx+3);
>         }
>     }
>     if (changes.indexOf("StreamUrl") >= 0) {
>         // Display album art
>         art.src = player.metaStreamUrl;
>     } else {
>         art.src = "";
>     }
> }
> </code>
>
> Artist: <span id="artist"></span><br>
> Track: <span id="track"></span><br>
> Album Art:<br>
> <img id="art">
>
> <audio src="http://an-internet-radio-station.whatever" id="player"
>     dynamicmetadata onmetadatachange="new_meta(ev)">
>
> </audio>
>

This is related to https://wicg.github.io/mediasession/, which is the API
for telling the browser about what the metadata is, so that it can be used
in UI that is outside of the page.

Two things seem to be missing, then. First,
http://www.smackfu.com/stuff/programming/shoutcast.html isn't detailed
enough to get interoperable implementations, in particular the metadata
keys would have to be defined.

Second, as already outlines above, one needs to be able to get the current
metadata somehow. I think that a mediaElement.getMetadata() method that
returns a https://wicg.github.io/mediasession/#the-mediametadata-interface
instance
would make sense, and then the metadatachange event could be a simple event
with no extra information on it.

There is a very old bug for exposing the metadata:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=5755

In order to make progress, there needs to be implementer interest. Although
it may well fizzle out, a new issue
https://github.com/whatwg/html/issues/new with the concrete suggested
changes for HTML would be a good starting point.
Received on Monday, 10 April 2017 04:45:28 UTC

This archive was generated by hypermail 2.3.1 : Monday, 10 April 2017 04:45:29 UTC