HTML Speech Incubator Group Final Report

3.1.1 Strong Interest

A requirement was classified as having "strong interest" if at least 80% of the group believed it needs to be addressed by any specification developed based on the work of this group. These requirements are:

• FPR40. Web applications must be able to use barge-in (interrupting audio and TTS output when the user starts speaking).
• FPR4. It should be possible for the web application to get the recognition results in a standard format such as EMMA.
• FPR24. The web app should be notified when recognition results are available.
• FPR50. Web applications must not be prevented from integrating input from multiple modalities.
• FPR59. While capture is happening, there must be a way for the web application to abort the capture and recognition process.
• FPR52. The web app should be notified when TTS playback finishes.
• FPR60. Web application must be able to programatically abort tts output.
• FPR38. Web application must be able to specify language of recognition.
• FPR45. Applications should be able to specify the grammars (or lack thereof) separately for each recognition.
• FPR1. Web applications must not capture audio without the user's consent.
• FPR19. User-initiated speech input should be possible.
• FPR21. The web app should be notified that capture starts.
• FPR22. The web app should be notified that speech is considered to have started for the purposes of recognition.
• FPR23. The web app should be notified that speech is considered to have ended for the purposes of recognition.
• FPR25. Implementations should be allowed to start processing captured audio before the capture completes.
• FPR26. The API to do recognition should not introduce unneeded latency.
• FPR34. Web application must be able to specify domain specific custom grammars.
• FPR35. Web application must be notified when speech recognition errors or non-matches occur.
• FPR42. It should be possible for user agents to allow hands-free speech input.
• FPR48. Web application author must be able to specify a domain specific statistical language model.
• FPR54. Web apps should be able to customize all aspects of the user interface for speech recognition, except where such customizations conflict with security and privacy requirements in this document, or where they cause other security or privacy problems.
• FPR51. The web app should be notified when TTS playback starts.
• FPR53. The web app should be notified when the audio corresponding to a TTS element is played back.
• FPR5. It should be easy for the web appls to get access to the most common pieces of recognition results such as utterance, confidence, and nbests.
• FPR39. Web application must be able to be notified when the selected language is not available.
• FPR13. It should be easy to assign recognition results to a single input field.
• FPR14. It should not be required to fill an input field every time there is a recognition result.
• FPR15. It should be possible to use recognition results to multiple input fields.
• FPR16. User consent should be informed consent.
• FPR18. It must be possible for the user to revoke consent.
• FPR11. If the web apps specify speech services, it should be possible to specify parameters.
• FPR12. Speech services that can be specified by web apps must include network speech services.
• FPR2. Implementations must support the XML format of SRGS and must support SISR.
• FPR27. Speech recognition implementations should be allowed to add implementation specific information to speech recognition results.
• FPR3. Implementation must support SSML.
• FPR46. Web apps should be able to specify which voice is used for TTS.
• FPR7. Web apps should be able to request speech service different from default.
• FPR9. If browser refuses to use the web application requested speech service, it must inform the web app.
• FPR17. While capture is happening, there must be an obvious way for the user to abort the capture and recognition process.
• FPR37. Web application should be given captured audio access only after explicit consent from the user.
• FPR49. End users need a clear indication whenever microphone is listening to the user.

3.1.2 Moderate Interest

A requirement was classified as having "moderate interest" if less than 80% but at least 50% of the group believed it needs to be addressed by any specification developed based on the work of this group. These requirements are:

• FPR33. There should be at least one mandatory-to-support codec that isn't encumbered with IP issues and has sufficient fidelity & low bandwidth requirements.
• FPR28. Speech recognition implementations should be allowed to fire implementation specific events.
• FPR41. It should be easy to extend the standard without affecting existing speech applications.
• FPR36. User agents must provide a default interface to control speech recognition.
• FPR44. Recognition without specifying a grammar should be possible.
• FPR61. Aborting the TTS output should be efficient.
• FPR32. Speech services that can be specified by web apps must include local speech services.
• FPR47. When speech input is used to provide input to a web app, it should be possible for the user to select alternative input methods.
• FPR56. Web applications must be able to request NL interpretation based only on text input (no audio sent).
• FPR30. Web applications must be allowed at least one form of communication with a particular speech service that is supported in all UAs.
• FPR55. Web application must be able to encrypt communications to remote speech service.
• FPR58. Web application and speech services must have a means of binding session information to communications.
• FPR6. Browser must provide default speech resource.
• FPR20. The spec should not unnecessarily restrict the UA's choice in privacy policy.

3.1.3 Mild Interest

A requirement was classified as having "mild interest" if less than 50% of the group believed it needs to be addressed by any specification developed based on the work of this group. These requirements are:

• FPR29. Speech synthesis implementations should be allowed to fire implementation specific events.
• FPR31. User agents and speech services may agree to use alternate protocols for communication.
• FPR43. User agents should not be required to allow hands-free speech input.
• FPR10. If browser uses speech services other than the default one, it must inform the user which one(s) it is using.
• FPR8. User agent (browser) can refuse to use requested speech service.
• FPR57. Web applications must be able to request recognition based on previously sent audio.

3.3.1 General Design Decisions

1. There are three aspects to the solution which must be addressed: communication with and control of speech services, a script-level API, and markup-level hooks and capabilities.
2. The script API will be Javascript.
3. The scripting API is the primary focus, with all key functionality available via scripting. Any HTML markup capabilities, if present, will be based completely on the scripting capabilities.
4. Notifications from the user agent to the web application should be in the form of Javascript events/callbacks.
5. For ASR, there must at least be these three logical functions:
   1. start speech input and start processing
   2. stop speech input and get result
   3. cancel (stop speech input and ignore result)
6. For TTS, there must be at least these two logical functions:
   1. play
   2. pause
   There is agreement that it should be possible to stop playback, but there is not agreement on the need for an explicit stop function.
7. It must be possible for a web application to specify the speech engine.
8. Speech service implementations must be referenceable by URI.
9. It must be possible to reference ASR grammars by URI.
10. It must be possible to select the ASR language using language tags.
11. It must be possible to leave the ASR grammar unspecified. Behavior in this case is not yet defined.

3.3.2 Speech Service Communication and Control Design Decisions

This is where design decisions regarding control of and communication with remote speech services, including media negotiation and control, will be recorded.

3.3.3 Script API Design Decisions

This is where design decisions regarding the script API capabilities and realization will be recorded.

• It must be possible to define at least the following handlers (names TBD):
  • onspeechstart (not yet clear precisely what start of speech means)
  • onspeechend (not yet clear precisely what end of speech means)
  • onerror (one or more handlers for errors)
  • a handler for when the recognition result is available
  Note: significant work is needed to get interoperability here.

3.3.4 Markup API Design Decisions

This is where design decisions regarding the markup changes and/or enhancements will be recorded.