W3C home > Mailing lists > Public > public-xg-htmlspeech@w3.org > October 2011

<reco> examples

From: Michael Bodell <mbodell@microsoft.com>
Date: Thu, 27 Oct 2011 11:46:27 +0000
To: "public-xg-htmlspeech@w3.org" <public-xg-htmlspeech@w3.org>
Message-ID: <22CD592CCD76414085591204EB19F4E8239ADFB4@TK5EX14MBXC262.redmond.corp.microsoft.com>
I updated a simple reco example in the latest web api document and tried to add more text about how binding would work to make more concrete the proposal.  Glen also sent out some reco examples at http://lists.w3.org/Archives/Public/public-xg-htmlspeech/2011Oct/0048.html that are well worth reviewing and while it does require flash I quite like Charles's examples from http://lists.w3.org/Archives/Public/public-xg-htmlspeech/2011Oct/0055.html and think it would also be a clear place where markup can solve the task.

I know there has been some discussion the past few weeks about a "Speech IME" that Satish and some others suggested may negate the need for the <reco> tag (because the UA will use a Speech IME to speech enable all input fields) but I fundamentally disagree.  The model of a speech IME in the UA means the web application author does not have the abiltiy to select either the grammars nor the recognition service.  In essence the web application author is at the mercy of the UA for the speech web service.  That is fundamentally unexceptable.  The web application author needs the control over choosing which recognition service is in place.

There have been some other discussions about decalative versus procedural and here to I agree with Glen.  There are a wide variety of users.  Some only know mark up and want to copy and paste mark up from view-source and want it to work.  Some know mark up plus very basic JS to write conditions and handlers in attributes of elements.  Some live and breath a dynamic JS web where all of their elements are constructed in JS.  We should endevour to make the easy truely easy (and for many people that means mark up with a minimum of JS) while leaving the hard possible (and for some things that does mean you need to go to scripting).

Here is my take on the 8 examples from Glen's original Oct 4 mail http://lists.w3.org/Archives/Public/public-xg-htmlspeech/2011Oct/0000.html

Example 1: Using <reco> with a simple field

<form>
  <reco/>
  <input id="in1" type="text"/>
</form>

The reco inside a form is automatically is associated with the next input.  The default grammars work with the text and the default assignement is what we want.  If you wanted to be explicit with the for attribute and also wanted to make sure the value was always overwritten you could do:

<form>
  <reco for="in1" onresult="in1.value = this.item(0).interpretation"/>
  <input id="in1" type="text"/>
</form>

Note that with the semantics that are in the current proposal these two will be slighlty different in the case that there was some text already in the input.  Consider that the user typed "this is speech" and then highlighted the "is" before speaking and that the interpretation of the speech is the word "cool".  The first markup would now have the value of the input in1 as "this cool speech" while the second one would be just "cool".

Example 2: Using CSS to position reco
<form>
  <reco style="position:relative; left:-30px;"/>
  <input id="in2" type="text"/>
</form>

CSS still works the same, although there is no guarentee the UA will work with the style the way you are expecting as the UA rendering of the reco element could be producing a button in the chrome that wouldn't be styled like this.  But in any case, normal CSS rules would work.

Example 3: Overwriting and then submitting the form:
<form action="http://google.com/search" method="get">
  <reco onresult="q.value=this.item(0).interpretation; this.form.submit()"/>
  <input id="q" type="text"/>
</form>

No problem to define the simple 2 JS statements as an attribute on reco.

Example 4: Using <reco> to append continuous text to <input> field
<form>
  <reco continuous="true"/>
  <input id="in3" type="text"/>
</form>

Here the smart default binding will be appending to the input anyways.

Example 5: Using <reco> to append continuous text to <textarea>
<reco continuous="true" for="ta"/>
<textarea id="ta" cols="50" rows="2"></textarea>

Again the binding by default appends.

Example 6: Using <reco> to append continuous text to <span>
<span id="spn" style="border:solid red;"></span>
<reco continuous="true" onresult="spn.innerHTML+=this.item(0).interpretation"/>

This time no for (neither implicit nor explicit) but the tag still works (assuming the UA will do something smart with the default grammar and with the visual rendering of this reco control).

Example 7: Using <reco> to append continuous text to <div>
<div id="dv" style="border:dashed blue;"></div>
<reco continuous="true" onresult="dv.innerHTML+=this.item(0).interpretation"/>

Same as above.

Example 8: Using <reco> to select from a menu
<form>
  <reco/>
  <select id="sel">
    <option value="asparagus">asparagus</option>
    <option value="broccoli">broccoli</option>
    <option value="cauliflower">cauliflower</option>
    <option value="spinach">spinach</option>
  </select>
</form>

Here the reco inside the form will be implicitly set to the select element in the form, the default assignment will be if the option matches to set the selectedness to true.  The one caveat that should be handled better would be the building of the grammar as right now it will just use builtin:select where it should probably do something like builtin:select?value=asparagus&value=broccoli&value=cauliflower&value=spinach or some other way of communicating the select options.  If I use a little more non-default behavior I can use the sample grammar Glen did and get:
<form>
  <reco for="sel" grammar="http://example.com/menugrammar.xml"/>
  <select id="sel">
    <option value="asparagus">asparagus</option>
    <option value="broccoli">broccoli</option>
    <option value="cauliflower">cauliflower</option>
    <option value="spinach">spinach</option>
  </select>
</form>

In general I think these type of mark up bindings are both simple and also quite powerful.  HTML 5 has already gone to the trouble to define a whole bunch of types, define what are valid text values or not, and say how to convert from text to the various types.  I think we'd be foolish to not leverage this.  If we do another pass or two we may get all the bindings specified perfectly, but even if we don't, I think an editors note in the final report would be valuable and the working group that fully standardizes our proposals could complete the details to make sure the i's are dotted and the t's are crossed correctly.
Received on Thursday, 27 October 2011 11:46:58 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 27 October 2011 11:46:58 GMT