Munich, 26-27 March 2007
Present: Marc Schröder (Chair, Scribe), Hannes Pirker, Jean-Claude Martin, Felix Burkhardt, Amaryllis Raouzaiou and Paraskevi Tzouveli (jointly standing in for Kostas Karpouzis), Björn Schuller (Host), Enrico Zovato, Jianhua Tao, Christian Peter
Table of Contents
Framing the group's activities: Identifying target users
Revisions of the structure of the requirements document
Relation of use case requirements to existing markup specifications
Prospects of continuing the group's work in the recommendation track
Appendix: Lists compiled at the meeting
List of relevant emotion descriptions from emotion theories
Representations of emotion in current affective computing systems
Christian: we should not try to please everyone, but narrow down. E.g., “technological context”.
Jean-Claude: we need a language which is simple enough, covering the main requirements.
Marc: we should show how our language is capable of representing scientifically valid emotion descriptions, to gain credibility.
Hannes: We have not yet talked much about the emotions themselves; mostly about additional information needed. That is probably inevitable, because there is no agreement on how to represent emotions. But we should ask ourselves: what would be the added value for someone to use this coding scheme?
ACTION: Jean-Claude, Christian and Jianhua to add links to example for each point in the requirements document: link to a use case example where this feature is required, by 27 April
A number of smaller modifications were discussed and agreed. These include, notably:
Addition of a section “1.0: Type of emotion-related phenomenon” -- mood, attitude, full-blown emotion, interpersonal stance, etc.
Merge “1.7 Internal vs. external state” into “1.6 Regulation” at the beginning, as the core property/a major requirement of regulation
Clarify that 1.9 describes the timing of the emotion proper – where in time is the emotion located; how does it evolve over time. Section 2 should provide timing for the following semantics of links to media: “observable behaviour” and “trigger”.
Section 1 should be split into a section on “Core emotion information” and a section on “Meta-information on emotion” (to be distinguished from “Global metadata”, which is at the document level). The section on “Meta-information on emotion” should contain the following subsections of the current section 1: “1.8 Acting”, “1.10 Confidence/Probability”, and “1.11 Modality”.
ACTION: Marc to restructure (again) the consolidated use case requirements as agreed, by 13 April.
Presentations:
HUMAINE database annotation scheme (HUMAINE D5g, pilot database in the toolbox) (Jean-Claude)
HUMAINE EARL proposal and specification (Marc)
IMDI: ISLE Meta Data Initiative (rich metadata spec, editing tools, open source)(Jean-Claude)
W3C InkML (trace representation; relative and absolute time) (Jean-Claude)
W3C EMMA (many links to recognition-related concepts) (Enrico)
HumanML (a very ambitious attempt in 2001 which seems to have failed) (Felix)
xml vs. rdf (rdf caters for semantics) (Amaryllis)
ACTION: Marc to collect all slides and upload them in the restricted space of the emoxg group
Volunteers to discuss existing ML in view of our requirements document (after it has been restructured again by Marc):
Enrico volunteers for EMMA and SSML
Hannes volunteers for SMIL
Björn volunteers for the HUMAINE EARL
??? for the HUMAINE Database annotation scheme
??? for IMDI
??? for InkML
Format of this discussion:
For each requirements section, point to the language element which (partially) addresses the requirement; possibly, a short comment.
Deadline: 4 June (i.e., before the next f2f)
We would like to see this work continue; either as a separate WG or as part of MMI or Voice Browser group. Paolo Baggia recommends we become part of an existing WG.
ACTION: Marc to liaise with W3C representatives about options and consequences of this kind of choice
Interest to continue from
Fraunhofer
CAS
Loquendo
NTUA
Deutsche Telekom
DFKI
Other W3C members might be interested in joining a standardisation effort for an emotion markup. Various ideas were discussed about whom to contact.
Final Report will be the main topic of the f2f in Paris.
Ethical issues: The Final Report should contain a section on ethical issues.
Steps towards a Final Report:
List emotion descriptions from emotion theory that we want to address
Link requirements to existing solutions in various ML
Formulate a number of examples for some types of markup (e.g., in XML or in N3) (Felix volunteers)
ACTION: Felix to formulate a number of examples for some types of emotion markup before the next phone meeting
Publications:
ACII (deadline 5 April)
use cases and resulting requirements
(Christian, Hannes, Felix, Jianhua, Enrico cannot contribute to this one unless the deadline is shifted; available: Jean-Claude, Laurence, Björn, Marc, ?Kostas?, ?Catherine?, ?Ian?)
HUMAINE Handbook (deadline: ...soon...)
“Scientifically valid....”
need to contribute: Marc, Hannes, Myriam; would like to contribute: Felix, ?Kostas?, Björn, Jianhua; can only contribute after 8 May: Christian; rather not: Enrico
Springer book: Emotion and Affect in HCI (Christian – deadline 25 June)
Section “Community: Standardisation efforts”, 10-14 pages
Intention: Overview of the activities in the field of standardisation
Springer book: Affective Information processing (Jianhua – draft deadline: 30 August)
could summarise the entire work of the XG, or go beyond it (final deadline in October == some WG work could be included)
Next phone meeting: 23 April, 11:00 UTC (1pm in Germany)
Felix suggested that we make much more use of the IRC facility in future phone meetings. Felix reported of other W3C groups he has been in: people basically type (a short version of) what they say. All present expressed that it should be done like this in the future – it will help understanding each other and it will simplify the scribe's job.
Date of next f2f: 7 June 2007, Paris. Many of the participants need to check if they can come.
(copied from above)
ACTION: Marc to restructure (again) the consolidated use case requirements as agreed, by 13 April.
ACTION: Jean-Claude, Christian and Jianhua to add links to example for each point in the requirements document: link to a use case example where this feature is required, by 27 April
ACTION: Marc to collect all slides and upload them in the restricted space of the emoxg group
ACTION: volunteers to discuss existing ML in view of our requirements document by 4 June
ACTION: Felix to formulate a number of examples for some types of emotion markup before the next phone meeting
ACTION: volunteers to contribute to publications, before the respective deadlines
ACTION: Hannes to ask Tanja to point us to a short list of the most important emotion theories and associated emotion descriptions that we should definitely cover in view of a technological context.
ACTION: Hannes to invite Tanja to the f2f in Paris.
Basic emotions, and other lists of emotion words
Emotion dimensions
PAD, etc. (Pleasure=Valence=Evaluation; Arousal=Activity=Activation; Control=Dominance=Power)
Grid study (add Unexpectedness)
Appraisal-based / cognitive emotion theories
OCC
Scherer's appraisal model with / without sequence in time
Complex emotion according to Ekman 75, Scherer 98, Plutchik, Izard 94, ...
combination of several emotions
Regulation model (Gross, 2007)
We think it should be possible to express the above in the markup language.
We should ask a psychologist for feedback on this; for example Tanja Bänziger. The goal is to point out obvious ommissions, important things we may have missed.
ACTION: Hannes to ask Tanja to point us to a short list of the most important emotion theories and associated emotion descriptions that we should definitely cover in view of a technological context.
ACTION: Hannes to invite Tanja to the f2f in Paris.
Emotion definition according to Scherer: multiple components:
We do not attempt to define these properties for the emotional states annotated in EmotionML. Instead, an application should define the kinds of properties meaningful to it. |
UC1
emotion categories
dimensions
appraisals
UC2
emotion categories and intensity
coordinates in n-dimensional emotion space including intensity
complex emotions (ambiguity and combination of emotions)
emotion-related states, e.g. stress, fatigue
UC3
OCC for generation
Scherer's appraisal model for generation
emotion categories and intensity for facial expression and speech
emotion dimensions for speech
We could start the final report with this kind of overview – it motivates why it would be inappropriate to try and come up with one description of emotions only.