DOM and events handling in VXML for a better multimodal application

Dear all

as discussed recently in the SpeechTek Europe 2011 event, i'm here to 
discuss about what will be very useful for having great multimodal 
application.

As you know better than me, a multimodal application is an application 
that is allowing the user to interact with the application cross-channel 
in the same time...BUT we have some tecnical and standards limitation 
today to do this in a safe and clear way.

I will show you an example:

I have a multimodal application allowing the user to interact with voice 
and web channels, and i have a conceptual "call flow" similar to this:

    * User call the application
          o VUI: "Hi Bob, how can i help you?"
          o GUI: "some html showing a tag cloud with FAQ"
    * User says "i want to make an flight reservation"
          o VUI: "please, click on the map over your destination or just
            say the name of the city where do you want to go"
          o GUI: "some html showing a map"


In this case, if the user click on the map using the GUI channel, i need 
to change immediately the VUI interface, but tecnically the VXML 
platform doesn't allow to stop the prompt (well, it depends on the 
bargein settings but in any case you need some ASR events to stop the 
prompt).

My conceptual solution for this problem, will be something like this:

in the VXML page (using javascript) start an AJAX HTTP request to a web 
page that will be return an xml response if any GUI event happends.

If i get an ASR event (NoInput/NoMatch/Match) the VXML page will post 
result to the submit page and the VXML application will work normally.

if, during the prompt (or generally, before to get any kind of ASR 
event) the AJAX request gets a response i will be able to throw a 
javascript event on the DOM structure, and here is where i think that 
the standard is missing: there should be some events (not necessary 
related to ASR recognition) capable to stop the ASR/TTS platform and 
force the submit of the form, even if ANY ASR events happends.

What do you think about that?

Regards

-- 
Nicola Mosca

***

IM Service Lab s.r.l. -
www.imservicelab.com