Packaging WebVTT with Audiobooks

Along the lines of my last email about experiments I’ve done recently with WebVTT for text/audio sync [1], I am also curious about how this approach could be useful in W3C Audiobooks [2].

The Audiobooks spec already mentions using HTML as an alternate for an audio resource [3]. I would add the following to introduce synchronization on top of that:

1. List the HTML and WebVTT file as alternates

"url": “cinderella.mp3#t=28.51,1162.22",
"name": "Cinderella; or, The Little Glass Slipper",
"alternate": [{
        "type": "LinkedResource",
        "url": “cinderella.vtt",
        "encodingFormat": "text/vtt"
},
{
        "type": "LinkedResource",
        "url": “cinderella.html",
        "encodingFormat": "text/html"
}]

You could even enable synchronization directly in the HTML file by adding the audio reference and the WebVTT reference there too:

<body>
    <audio src="cinderella.mp3#t=28.51,1162.22">
        <track kind="metadata" default src=“cinderella.vtt">
    </audio>
    ...
    <p>The chapter contents ...</p>
    …
   <script src=“sync-highlight.js">
</body>


Or you could say that it’s up to the user agent to make that association by using the data in the audiobook manifest to create an audio element on the fly and attach the WebVTT track to it, including some scripting attached to VTT cue events.

And, it’s worth nothing that by including the WebVTT track in the manifest as an “alternate”, you could use the WebVTT cues like phrase markers even with no HTML file. There is a type of DAISY book out there that has historically been very commonly produced, called “Audio + TOC”, that has no markup or text outside of the table of contents. It’s a table of contents plus audio files plus phrase markers (in SMIL) for the audio files. So using the WebVTT files in this way would mean that those DAISY books could fit into this format without losing information. 

Any thoughts, opinions, feelings? 

Marisa

1. https://lists.w3.org/Archives/Public/public-sync-media-pub/2021Nov/0000.html <https://lists.w3.org/Archives/Public/public-sync-media-pub/2021Nov/0000.html> 
and 
https://daisy.github.io/accessible-books-on-the-web/demos/moby-dick/chapter_001.html <https://daisy.github.io/accessible-books-on-the-web/demos/moby-dick/chapter_001.html>
2. https://www.w3.org/TR/audiobooks <https://www.w3.org/TR/audiobooks>
3. https://www.w3.org/TR/audiobooks/#example-13-audiobook-with-alternate-text <https://www.w3.org/TR/audiobooks/#example-13-audiobook-with-alternate-text>

Received on Monday, 22 November 2021 22:35:37 UTC