- From: Shires, Glen <glen.shires@intel.com>
- Date: Wed, 1 Oct 2003 18:47:10 -0400
- To: <www-multimodal@w3.org>
Here are some comments from the SALT Forum on the EMMA Working Draft at
http://www.w3.org/TR/2003/WD-emma-20030811/.
In SALT, speech recognition results are represented in an XML result
format. We hope that, once EMMA becomes a Recommendation, the SALT Forum
will be able to consider EMMA with a view to recommending it as the
standard result format for SALT environments.
1) First of all we would like to congratulate the EMMA team for
producing a well thought-out specification. EMMA includes standard
annotations for the main features of a speech recognition result that we
believe will be most useful in applications, notably:
- emma:tokens (for the spoken words of an utterance)
- emma:confidence (for the recognizer's confidence in an utterance)
- emma:one-of (for N-Best alternates).
2) We believe the syntax which places annotations inline in the
interpretation (Syntax 1 in the Introduction) is clean and compact and
we strongly support this format of annotation.
3) We have serious concerns about using the RDF syntax, notably:
i) increased verbosity
ii) difficulty/non-determinism of structural queries such as XPath
iii) extra query mechanism for application authors to learn.
These points are explained and illustrated below.
3.1) increased verbosity of EMMA documents
All the samples in the spec show that the inline syntax is more compact.
For applications where EMMA is expected to contain many annotations,
the divergence will be significant and likely to impact performance.
3.2) difficulty of querying RDF annotations in non-RDF environments
In SALT, as in many other XML environments, access to XML data such as
recognition results is through XPath queries on the XML structure of the
document. But if EMMA were to adopt the RDF syntax, such structural
queries would not only be more difficult to author, they would be
non-deterministic.
In illustration of this point, we have converted an example from the
SALT spec into the different EMMA syntaxes. These are reproduced at the
foot of this message. The original NLSML from the SALT spec is at [1].
The inline EMMA variant is at [2] and the RDF EMMA variant is at [3].
The binding query in the inline case is simple and transparent:
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//dest_city[@confidence > 0.4]" />
The bind query in the RDF case is more complex. To query a confidence
score, the test has to look for the "rdf:Description" node which is both
(a) about the semantic value in question, and (b) holds the correct
annotation on which the test is based:
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//rdf:Description[@rdf:about='#emma(id(\'int1\')/dest_city)' &&
@emma:confidence > 0.4]" />
This complexity applies beyond just conditional test attributes: it
applies to any binding statement which requires access to the
annotation. So when copying the confidence score into an HTML field, the
bind query is something like:
<salt:bind targetelement="txtConfidence"
value="//rdf:Description[@rdf:about=
'#emma(id(\'int1\')/dest_city)']/@emma:confidence" />
Similar concerns apply when the EMMA result is handled in script -- the
XML DOM navigation necessary to find the correct annotation would be
similarly complicated, unless the EMMA environment were to implement
both (a) an RDF processor, and (b) the emma() mechanism to support the
XPointer-derived query module (Section 2.1.2.1). This appears to be a
considerable implentational burden.
Secondly, the annotations of a single EMMA document can be represented
in a number of different ways in the RDFsyntax. For example, the
annotation:
<rdf:Description rdf:about="#emma(id('int1')/dest_city)"
emma:confidence="0.35" />
could equally be expressed:
<rdf:Description rdf:about="#emma(//emma:interpretation/dest_city)"
emma: confidence="0.35" />
This renders XPath queries non-deterministic, since a different binding
query to [2] would be required:
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//rdf:Description[@rdf:about='#emma(//emma:interpretation/dest_cit
y )' && @emma:confidence > 0.4]" />
Given that the application author may not know the preferences of the
grammar writer/recognition engine in constructing the annotational
tree, XPath queries on annotations in an EMMA RDF document are
effectively hit-and-miss.
3.3) Since, in either syntax, access to the instance data within the
interpretation is expected to be via a structural query such as XPath,
the introduction of a different mechanism for annotation data means that
the application author has to learn the new access mechanism.
In summary, EMMA in RDF would not be usable in SALT or other XPath
environments. We do not believe that the value added by RDF outweighs
the cost of implementing RDF. The costs we have noted are in the
implementation of the EMMA environment, the performance of EMMA systems,
and the burden on the application author.
Glen Shires, Chair, SALT Forum Technical Working Group
on behalf of the SALT Forum
(http://www.saltforum.org/members/members.asp)
=================
[1] Example
(from section 2.2.1.2 of the SALT spec.)
The original NLSML example and corresponding SALT bind:
<result grammar="http://flight"
xmlns:xf="http://www.w3.org/2000/xforms">
<interpretation confidence="0.4">
<input mode="speech">
I'd like to travel from Seattle to Boston
</input>
<xf:instance>
<airline>
<origin_city confidence="0.45">Seattle</origin_city>
<dest_city confidence="0.35">Boston</dest_city>
</airline>
</xf:instance>
</interpretation>
</result>
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//dest_city[@confidence > 0.4]" />
[2] Corresponding EMMA (inline) document and <bind>
<emma:emma emma:version="1.0"
xmlns:emma="http://www.w3.org/2003/04/emma#">
<emma:interpretation emma:process="http://flight"
emma:confidence="0.4"
emma:tokens="I'd like to travel from Seattle to Boston">
<airline>
<origin_city emma:confidence="0.45">Seattle</origin_city>
<dest_city emma:confidence="0.35">Boston</dest_city>
</airline>
</emma:interpretation>
</emma:emma>
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//dest_city[@emma:confidence > 0.4]" />
[3] Corresponding EMMA (RDF) document and <bind>
<emma:emma emma:version="1.0"
xmlns:emma="http://www.w3.org/2003/04/emma#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<emma:interpretation emma:id="int1">
<airline>
<origin_city>Seattle</origin_city>
<dest_city>Boston</dest_city>
</airline>
</emma:interpretation>
<rdf:Description rdf:about="#int1" emma:process="http://flight"
emma:tokens="I'd like to travel from Seattle to Boston"
emma:confidence="0.4" />
<rdf:Description rdf:about="#emma(id('int1')/origin_city)"
emma:confidence="0. 45" />
<rdf:Description rdf:about="#emma(id('int1')/dest_city)"
emma:confidence="0.35" />
</emma:emma>
<salt:bind targetelement="txtBoxDest"
value="//dest_city"
test="//rdf:Description[@rdf:about='#emma(id(\'int1\')/dest_city)' &&
@emma:confidence > 0.4]" />
Received on Thursday, 2 October 2003 11:57:05 UTC