Emotion Intensity in Multimodal Dialogue Datasets

This presentation explores emotion intensity analysis in multimodal dialogue systems, moving beyond categorical emotion detection to quantify emotional strength. A multimodal approach combining text and audio is explored because emotional intensity is conveyed through both linguistic content and acoustic features. A survey of around 20 dialogue datasets reveals that only IEMOCAP and MEISD simultaneously provide multimodal data, emotion intensity annotations, and dialogue structure. The primary focus is IEMOCAP, containing 151 dyadic dialogues with 10,086 utterances annotated for Valence-Arousal-Dominance (VAD) dimensions. Two research directions are proposed: detecting emotion intensity for current utterances given dialogue context, or predicting emotional intensity for future responses. The long-term goal is enabling emotion intensity-conditioned speech generation for more naturalistic voice agents.