Using the EMMA 2.0 "output" element to manage content complexity

EMMA (Extensible MultiModal Annotation) 1.0 was originally designed to represent semantically complex user input to interactive multimodal systems. EMMA 2.0 [1] introduces the ability to represent system output, in addition to user input. This makes it possible for both sides of a computer-human dialog to be represented in a uniform format. Many of the features originally designed for representing input also work well for representing output. In particular, the ability to represent alternatives (<emma:one-of>) may be useful for users with cognitive disabilities.

1. Users with cognitive disabilities may have difficulty understanding some complex text but may be able to understand the ideas conveyed by the text if they can read an alternative, simpler, version of the text.
2. In order to support users with different abilities to read text, authors could provide several alternative texts in an EMMA document, in <emma:output> elements within an <emma:one-of> tag, to support users who need simpler text.
3. For example, in a personal assistant application, the user could ask a question like "what will the weather be like today?". The system response could include several different alternative outputs, with different levels of complexity, which could be selected for different users, as shown in the example below.
4. In order for the user's coga assistive technology to be able to select the right level of complexity, the EMMA document could characterize the alternatives with a (hypothetical) "complexity" element (constructed for this example, and not part of EMMA). The complexity values in the examples below are grade levels in the Flesch-Kincaid [2] text complexity metric. (Flesch-Kincaid is a commonly used measure of text complexity, although there are many others.)
5. The user's coga assistive technology could select alternative texts by looking for a complexity less than a certain value, according to the user's preferences. Initially, the alternative presentations would have to be at least to some extent manually authored, but as text simplification technology improves they might eventually be generated automatically.
6. It's also possible to provide an image alternative, in case some users don't want any text at all, as in the fourth alternative.
7. The message also could be rendered with text to speech as another alternative presentation (not shown).

Note that EMMA 2.0 is still a Working Draft, so there is ample opportunity to provide suggestions and comments.

Here's an example of an EMMA document that could be used to provide four different alternative formats of a message about today's weather. The first three alternatives are texts with different levels of complexity, and the fourth is an image-only presentation.

          <emma:emma version="2.0"
                       xmlns:emma="http://www.w3.org/2003/04/emma"
                <emma:one-of id="ooo1"
                             emma:medium="visual"
                             emma:mode="text"
                             emma:result-format="text/html"
                             emma:lang="en=US"
                             <emma:output emma:confidence="0.8" id="html1">
                                 <complexity value="10.2" metric="Flesch-Kincaid"/>
                            <p>
                                Showers, mainly before 1pm. High near 50. Breezy, with a northwest wind 13 to 22 mph, 
                                with gusts as high as 32 mph. Chance of precipitation is 80%.
                            </p>
                    </emma:output>
                    <emma:output id="html2">
                         <complexity value="3.6" metric="Flesch-Kincaid"/>
                            <p>
                                It will probably rain in the morning. 
                                The high temperature will be about 50 degrees. It will be breezy.
                                The wind will come from the northwest, with gusts as high as 32 miles per hour.
                            </p>
                    </emma:output>
                    <emma:output id="html3">
                         <complexity value="2.4" metric="Flesch-Kincaid"/>
                            <p>
                                It will probably rain in the morning. It will be a little cool. It will also be
                                windy.
                            </p> 
                    </emma:output>
                    <emma:output id="html4">  emma:result-format="image/svg+xml"
                            <img>
                                src="weather.svg" alt="wind, rain, coat"
                            </img> 
                    </emma:output>
                </emma:one-of>
            </emma:emma>
    

Alternative Renderings

1. Complexity=10.2

   Showers, mainly before 1pm. High near 50. Breezy, with a northwest wind 13 to 22 mph, with gusts as high as 32 mph. Chance of precipitation is 80%.

2. Complexity=3.6

   It will probably rain in the morning. The high temperature will be about 50 degrees. It will be breezy. The wind will come from the northwest, with gusts as high as 32 miles per hour.

3. Complexity=2.4

   It will probably rain in the morning. It will be a little cool. It will also be windy.

4. image only

   

References

[1] EMMA 2.0

[2] Flesch-Kincaid Grade Level