Re: Synchronized narration work from the Sync Media Pub CG

Hi Chaals,

Some background: the primary use-case for "synchronized narration"
historically originates from DAISY Digital Talking Books, which later on
influenced the design of EPUB3 Media Overlays (also for "read aloud"
experiences). Both Open Standards make use of SMIL to define; in a
declarative manner; the discrete contiguous synchronization points between
structured "text" documents (typically, HTML) and continuous media
resources (specifically, audio streams / files). The audio is usually
studio-quality prerecorded human narration, but in some high-volume
production scenarios the audio can be generated using Text To Speech (this
his not only cost-effective, it also benefits end-users who do not have
access to high-quality synthetic voices).

In this publishing context, content creators keep the main publication
documents (i.e. HTML files) separate from the synchronization "overlays"
(i.e. SMIL + audio), akin to the logical organization of external CSS
stylesheets (i.e. markup/semantics vs. presentation/styling). Publishers
have the option to publish a "text-only" publication, and/or its
"synchronized text+audio" variant. In the latter case, users have the
option to completely ignore the audio overlay (i.e. the "read aloud"
playback functionality is optional) and can consume the "text" of the
publication instead (which is equivalent to the original, unmolested HTML
files).

The "sync media" effort (specifically, "audio narration") aims to draft an
evolution of the aforementioned "talking books" publishing standards that
"aligns" with the W3C Web Publication proposals. For instance, JSON
encoding + URI Media Fragments (for timestamps) instead of SMIL XML, and a
metadata notation that is compatible with the WebPub model.

Please visit the following link (sorry, unformatted spec. draft) for
details on why no existing technology seems to meet the functional
requirements of "synchronized text/audio narration":
https://w3c.github.io/sync-media-pub/explainer.html

...and please let us know your thoughts!
Thanks, Daniel


On Wed, 12 Jun 2019 at 09:58, Charles 'chaals' (McCathie) Nevile <
chaals@yandex.ru> wrote:

> Hi,
>
> Thanks for the pointer.
>
> I'm curious why you wouldn't use e.g. WebVTT or another existing markup
> that carries the associations between text and audio renderings.
>
> cheers
>
> Chaals
>
> On Wed, 12 Jun 2019 02:12:26 +0200, Marisa DeMeglio <
> marisa.demeglio@gmail.com> wrote:
>
> Hi all,
>
> Chris Needham suggested that I share with you what we’ve been working on
> in the Synchronized Media for Publications CG - it’s a lightweight JSON
> format for representing pre-recorded narration synchronized with HTML, to
> provide an accessible reading experience. The primary use case is in web
> publications (we are involved in the Publishing WG), but it has been
> designed to live as a standalone “overlay” for HTML documents. Below are
> some links to the latest drafts.
>
> And, just to give you an idea of the basic user experience, here are two
> slightly different proof of concept demos for the sample in our repository:
>
> -
> https://raw.githack.com/w3c/sync-media-pub/master/samples/single-document/index.html
> -
> https://raw.githack.com/w3c/sync-media-pub/feature/custom-read-aloud-player/samples/single-document/index.html
>
>
> Interested in hearing your thoughts!
>
> Marisa DeMeglio
> DAISY Consortium
>
>
> Begin forwarded message:
>
> *From: *Marisa DeMeglio <marisa.demeglio@gmail.com>
> *Subject: **Drafts and a sample*
> *Date: *June 8, 2019 at 6:58:13 PM PDT
> *To: *W3C Synchronized Multimedia for Publications CG <
> public-sync-media-pub@w3.org>
>
> Hi all,
>
> As we discussed at the web publications F2F last month, we have some
> drafts up for review:
> https://w3c.github.io/sync-media-pub/
>
> Have a look specifically at the proposed synchronization format:
> https://w3c.github.io/sync-media-pub/narration.html
>
> And how to include it with web publications:
> https://w3c.github.io/sync-media-pub/packaging.html
>
> I’ve extracted the issues from our previous drafts and discussions, and
> put them in the tracker:
> https://github.com/w3c/sync-media-pub/issues
>
> I also started putting together a sample and playing around with some
> ideas for a simple proof-of-concept for playback:
> https://github.com/w3c/sync-media-pub/tree/master/samples/single-document
>
> (for anyone really interested and wanting to dig in: it needs to be more
> clever about how it uses the audio api - the granularity of timeupdate in
> the browser isn’t very good).
>
> Please feel free to comment, propose solutions, and otherwise share your
> thoughts.
>
> Thanks
> Marisa
>
>
>
>
>
> --
> Using Opera's mail client: http://www.opera.com/mail/
>