Use Case USC

The focus of our work is computational models of emotion. We are interested in them for two reasons. We use them both as a framework for exploring psychological theories of emotion as well as a key component of ECAs.  We. have 3 related uses for emotion annotations. 
1.	The cognitive-emotional state of our ECAs inform their behavior in a multi step process. First the communicative intent and cognitive-emotional state of the agent is conveyed via an XML functional markup to a behavior generation process. That process in turn specifies a behavioral plan (surface text, gestures, etc) using a xml-based behavioral markup. 
2.	There is a roughly inverse mapping process for the ECAs perception of other agents or humans. That is, there are recognition processes that map from the surface behavior of others to the behavioral markup and then map the behavioral markup to a functional markup. 
3.	We annotate videos of human behavior both in terms of observed behaviors and inferred emotions. This data collection effort informs and validates the design of our emotion model. In addition, the annotated video data contributes to the function and behavior mapping processes discussed above. 

Requirements on the kinds of markup we use.

o	We have separate functional and behavioral markups (we are following the SAIBA framework [Kopp et al. 2006]) for the multi-step process mentioned in 1.)
o	To maximally leverage the work across all the above uses, we want to use the same markup scheme across the uses.
o	The emotional state information in the functional markup follows the emotion theory we are modeling. The focus of our work is appraisal theory so our markup allows not only emotional category tags (e.g., fear) and intensity, but also the specification of causal antecedents of emotion -  appraisal variables such as desirability of the event, who is blameworthy for the event, etc. These appraisals all have an intensity. 
o	There are also more persistent affective information that can be conveyed from our model, such as trait affect and mood.
o	To do the behavioral markup, we need to know additional information on the appraisals such as who did the blameworthy act. Obviously, it has different behavioral consequences if the cause of the emotion is the addressee of an utterance or a third party that is not present, for example.
o	The behavior manifestations of felt emotions is distinct from intentionally communicated emotions that are not felt. In addition there are distinctions based on whether the emotion expression is being either suppressed or masked. So we need our markups support that distinction.
o	Different constituents of an utterance will have different emotions associated with them, so markups have to annotate those parts separately.
o	In our model, there may be multiple emotions in play at a time. Behaviorally, that suggests emotion blends.
o	Both emotions and the physical expressions of emotions have durations and dynamics. There is a question of the extent to which the dynamics should be encoded in the annotations as opposed to the emotion model that drives the annotations or the animation controllers that model the physical motion of the expression.