Re: Proposal to add start, stop, and update events to TTS

I've updated the spec by adding attribute SpeechSynthesisMarkerCallback
onmarker.
https://dvcs.w3.org/hg/speech-api/rev/dd316a766282

As always, the current draft spec is at:
http://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html

I did not add definitions for parameters DOMString markerName, unsigned
long charIndex, float elapsedTime because specific text has not yet been
proposed.

- Should markerName be changed from "DOMString" to "type ( enumerated
string ["word", "sentence", "marker"] )".  If so, how are named markers
returned? (We could add a "DOMString namedMarker" parameter or add an
"onnamedmarker() event".

- What should be the format for elapsedTime?

I propose the following definition:

SpeechSynthesisMarkerCallback parameters

charIndex parameter
The zero-based character index into the original utterance string of the
word, sentence or marker about to be spoken.

/Glen Shires


On Wed, Sep 19, 2012 at 8:44 AM, Glen Shires <gshires@google.com> wrote:

> Clearly callback functions that take arguments, such as for the 'onupdate'
> event, need to have explicitly specified types.  Unless there's any
> disagreements, I will add the 'onupdate' event and
> 'SpeechSynthesisUpdateCallback', as proposed above, to the spec on Friday.
>
> We may also want to add an argument to onstart, onend, onpause and
> onresume that indicates the object that fired it (particularly because a
> copy of the object is added to the queue by the 'speak' method).  (If I'm
> understanding correctly, the "attribute Function" alternative would require
> defining a different callback function to identify each object that gets
> added to the queue.)
>
> On Tue, Sep 18, 2012 at 3:38 PM, Dominic Mazzoni <dmazzoni@google.com>wrote:
>
>> Some clarification:
>>
>> 1. In other specs I saw that callback functions had explicitly
>> specified types, even if they take no arguments. But I have no
>> preference - if it's sufficient to just say Function onstart, then I'm
>> happy.
>>
>> 2. Rather than separate callbacks for word, sentence, character,
>> marker, etc. I suggested a single 'onupdate' callback with arguments
>> for marker, start time, charIndex, etc. - the reason is that different
>> speech engines support different types of updates, and trying to force
>> them all to have the same definition of "word", etc. is problematic.
>> This allows for future extensions to the API and also makes it easier
>> for clients to adapt to different speech engines.
>>
>> - Dominic
>>
>> On Tue, Sep 11, 2012 at 4:06 PM, Dominic Mazzoni <dmazzoni@google.com>
>> wrote:
>> > Sorry for being unclear. What I meant was to just extend the onstart
>> > and onend events already in the spec.
>> >
>> > - Dominic
>> >
>> > On Tue, Sep 11, 2012 at 2:48 PM, Glen Shires <gshires@google.com>
>> wrote:
>> >> Is this preferable to:
>> >>
>> >> interface SpeechSynthesisUtterance {
>> >>       ...
>> >>       attribute Function onstart;
>> >>       attribute Function onend;
>> >>
>> >>
>> >> On Tue, Sep 11, 2012 at 1:19 AM, Dominic Mazzoni <dmazzoni@google.com>
>> >> wrote:
>> >>>
>> >>> I propose adding a mechanism by which JavaScript clients can attach
>> >>> event listeners to an utterance to get notified when it starts
>> >>> speaking, when it finishes, and optionally with updates of the
>> >>> progress.
>> >>>
>> >>> Here's what could be added to the spec:
>> >>>
>> >>> callback SpeechSynthesisStartCallback = void();
>> >>> callback SpeechSynthesisEndCallback = void();
>> >>> callback SpeechSynthesisUpdateCallback = void(DOMString markerName,
>> >>> unsigned long charIndex, float elapsedTime);
>> >>>
>> >>> interface SpeechSynthesisUtterance {
>> >>>   ...
>> >>>   attribute SpeechSynthesisStartCallback onstart;
>> >>>   attribute SpeechSynthesisEndCallback onend;
>> >>>   attribute SpeechSynthesisUpdateCallback onupdate;
>> >>> }
>> >>>
>> >>> I propose that "onstart" and "onend" must be supported for an
>> >>> implementation to be fully compliant. There are too many applications
>> >>> that can't be implemented without these.
>> >>>
>> >>> I think "onupdate" should be optional because it depends on what's
>> >>> possible in the speech engine. If the speech engine provides
>> >>> word-level information, for example, then every time there's a break
>> >>> between works it'd call onupdate() with the character index (into the
>> >>> original utterance string) and the elapsed time since speech began. An
>> >>> engine might also notify when certain named "markers" (e.g. in SSML)
>> >>> are reached.
>> >>>
>> >>> Other ideas - in the Chrome TTS extension API I also implemented these
>> >>> events - what do you think?
>> >>>
>> >>> * Error
>> >>> * Cancelled (stopAndFlushQueue called before it ever started playing)
>> >>> * Interrupted (stop called while it was in the middle of speaking)
>> >>>
>> >>> - Dominic
>> >>>
>> >>
>>
>
>