On Emotions as Features for Speech Overlaps Classification

Abstract

Listen

Translate article

Although being a frequently occurring phenomenon in spoken communication, speech overlaps did not obtain the deserved attention in research so far—in both Human-Human Interaction (HHI) and Human-Computer Interaction (HCI). It is common knowledge that overlaps can figure as a competitive, rude interruption as well as a cooperative, convenient feedback signal giving important insight on the course of the interaction—but how are they related to the internal state of the overlapping speaker or the overlapped speaker? In this paper, we investigate dyadic human-human interactions and focus on the relations between the emotional changes occurring around overlaps in both interaction participants. Further to an in-depth statistical analysis of the changes in control and valence levels with respect to the nature of the overlap, we also present a classification approach based on features derived from such emotional changes surrounding an overlap and compare the classification performance of these features to classic acoustic features. We show that the automatic classification of competitive and cooperative overlaps using the changes in valence and control levels of the overlapping speaker outperforms common approaches employing acoustic and linguistic features.