Re: HTML and Multimedia Synchronization

Hi Adam,

Many of the things that you are stating as suggested features are already
possible with existing Web technologies in browsers. Have you tried
implementing a digital mathematics book with existing HTML5 technologies in
a Web browser? What limitations have you observed?

Regards,
Silvia.

On Fri, Jan 4, 2013 at 5:07 AM, Adam Sobieski <adamsobieski@hotmail.com>wrote:

> Silvia Pfeiffer,
>
> The email was broad, introducing concepts for discussion, suggestions for
> features for HTML. Concretely, the suggested features are multimedia
> synchronization with speech synthesis and with 3D graphics, WebGL or X3DOM.
> The functionality can be achieved with XHTML and JavaScript; HTML could
> come to include multimedia synchronization features; additionally, a new
> HTML+SMIL document could be a deliverable. The interoperabilities between
> multimedia synchronization, JavaScript, speech synthesis, and 3D graphics
> are suggested features.
>
> A specific example includes a digital mathematics book or textbook with a
> page of hypertext and a 3D graphic or widget which renders 3D graphics with
> multitouch ergonomics. In the example, the 3D graphic is a mathematical
> function and some hypertext on the page discusses the mathematical
> function. The hypertext content contains hyperlinks which activate
> rotation, scaling, translation, and animations of the 3D graphic or objects
> in the 3D graphic.
>
> In the example, there is audio overlay interoperability. The page has one
> or more audio overlays available for users and, when a voice actor reads
> the content, the 3D graphic of the mathematical function rotates, scales,
> translates, and animates as if the hyperlinks were being clicked as the
> voice actor read each.
>
> In the example, there is also speech synthesis interoperability. When a
> speech synthesizer synthesizes the hypertext content, the 3D graphic of the
> mathematical function rotates, scales, translates, and animates as if the
> hyperlinks were being clicked as the synthesizer synthesized each.
>
> An example hypertext region:
>
> <p>{1}... as you can see in {2}, the <a href="{3}" {4}>...</a> of the
> function...{/1}</p>
>
> Region {1} is a sentence element, and could be a <span>, <ssml:s> in
> XHTML, or new element.
>
> Region {2} is hypertext to refer to or cross-reference a figure or 3D
> graphics element.
>
> Region {3} calls a JavaScript function.
>
> Region {4} is markup pertaining to multimedia synchronization including
> for audio overlays and speech synthesis.
>
>
>
> Kind regards,
>
> Adam
>