[EMOXG] XML suggestion for Meta2: Modality

Dear All,

This is my proposal for Meta2 in response to action 29 (http://www.w3.org/2005/Incubator/emotion/group/tracker/actions/29)


Meta 2: Modality
----------------

http://www.w3.org/2005/Incubator/emotion/XGR-requirements/#Modality
states:
"The emotion markup must be able to represent the modalities in which the emotion is reflected, e.g. face, voice, body posture or hand gestures, but also lighting, font shape, etc. The emotion markup must provide a mechanism to represent an open set of values."

To annotate modality, I'm proposing the optional tag <modality> as a subtag of the main emoML element (<emotion>). Since the requirement states that a flexible mechanism has to be adopted, a solution similar to those proposed for other core requirements could be exploited. This means that the EmoML should provide a default set of modality values, that could be extended or substituted by customizable sets. To this end, the <modality> tag should have the mandatory attributes 'set' and 'mode' that specify the reference modalities set and the modality itself.

A first proposal for the default set, whose name could be "basic_modalities", includes the following items:

basic_modalities, proposal 1

            - face
            - voice
            - body
            - text

Some of these values could be suitable both for verbal and non-verbal communication. For example, voice could be used for speech as well as for non linguistic events like laughs, sighs, etc. Body could be used for meaningless gestures as well as for signs. If we want to keep this distinction, a more detailed set could be adopted like in this second proposal:

basic_modalities, proposal 2

            - facial_expression
            - non_speech_sound
            - paralinguistic
            - body_sign
            - body_gesture
            - text
            - drawing


This set could be extended with further modalities in which emotions are reflected, like for example sound, music, lighting, color, shape, touch, etc.

Here is an example of annotation, with the default set "basic_modalities":

 <emotion>
     <category set="everyday" name="pleasure"/>
     <modality set="basic_modalities" mode="face"/>
 </emotion>

and, with a custom set:

 <emotion>
     <category set="everyday" name="pleasure"/>
     <modality set="my_modalities" mode="lighting"/>
 </emotion>


Regarding modality, I have found interesting the approach adopted by EMMA in which the distinction between 'medium' and 'mode' is proposed. 'medium' is a broader category that is used to specify the input channel. Three values are defined: "acoustic", "visual" and "tactile". 'mode' is an attribute with an open set of values that further specify the mode of communication (speech, dtmf,keys, video,etc.)
If we would like to include the "medium" attribute, the annotation becomes:

 <emotion>
     <category set="everyday" name="pleasure"/>
     <modality medium="visual" set="basic_modalities"  mode="face">
 </emotion>

or, in alternative, only the medium attribute could be used. In this case the "set" and "mode" attribute could be optional:

 <emotion>
     <category set="everyday" name="pleasure"/>
     <modality medium="visual">
 </emotion>



Multimodality
_____________


The emotion ML has to deal with multimodal inputs and outputs. The simplest way of representing more modalities is to annotate one element for each of them. In order to better classify and distinguish them, an identifier attribute could be introduced. Despite its verbosity, this representation is simple and clear. Here is an example:

<emotion>
     <category set="everyday" name="excited"/>
     <modality id="m1" set="basic_modalities" mode="face"/>
     <modality id="m2" set="basic_modalities" mode="voice"/>
</emotion>

A more compact representation could be obtained by introducing composite values separated by special characters. A similar solution has been proposed by Bjoern for Core 1, “Type of Emotion related phenomena” , variant 3b.
If we use the separator char '&':

<emotion>
     <category set="everyday" name="excited"/>
     <modality set="basic_modalities" name="face&voice"/>
</emotion>

Or, if multiple modality sets are allowed for the same annotation:

<emotion>
     <category set="everyday" name="excited"/>
     <modality set="basic_modalities&my_modalities" name="voice&music"/>
</emotion>


Regards,
Enrico




Gruppo Telecom Italia - Direzione e coordinamento di Telecom Italia S.p.A.

================================================
CONFIDENTIALITY NOTICE
This message and its attachments are addressed solely to the persons above and may contain confidential information. If you have received the message in error, be informed that any use of the content hereof is prohibited. Please return it immediately to the sender and delete the message. Should you have any questions, please send an e_mail to <mailto:webmaster@telecomitalia.it>webmaster@telecomitalia.it. Thank you<http://www.loquendo.com>www.loquendo.com
================================================

Received on Saturday, 9 August 2008 15:49:02 UTC