Common cues to emotion in the dynamic facial expressions of speech and song.

Steven R Livingstone,Marcelo M Wanderley,William F Thompson,Caroline Palmer

doi:10.1080/17470218.2014.971034

Steven R Livingstone, Marcelo M Wanderley + Show 2 more

Open Access

https://doi.org/10.1080/17470218.2014.971034

Copy DOI

Abstract

Speech and song are universal forms of vocalization that may share aspects of emotional expression. Research has focused on parallels in acoustic features, overlooking facial cues to emotion. In three experiments, we compared moving facial expressions in speech and song. In Experiment 1, vocalists spoke and sang statements each with five emotions. Vocalists exhibited emotion-dependent movements of the eyebrows and lip corners that transcended speech–song differences. Vocalists’ jaw movements were coupled to their acoustic intensity, exhibiting differences across emotion and speech–song. Vocalists’ emotional movements extended beyond vocal sound to include large sustained expressions, suggesting a communicative function. In Experiment 2, viewers judged silent videos of vocalists’ facial expressions prior to, during, and following vocalization. Emotional intentions were identified accurately for movements during and after vocalization, suggesting that these movements support the acoustic message. Experiment 3 compared emotional identification in voice-only, face-only, and face-and-voice recordings. Emotion judgements for voice-only singing were poorly identified, yet were accurate for all other conditions, confirming that facial expressions conveyed emotion more accurately than the voice in song, yet were equivalent in speech. Collectively, these findings highlight broad commonalities in the facial cues to emotion in speech and song, yet highlight differences in perception and acoustic-motor production.

Highlights

Throughout history, speech and song have served as overlapping and interchangeable forms of vocal expression
A significant Statement × Emotion interaction was reported for 300 ms of the epoch containing vocal sound, F (12, 132) = 3.72, p, .05, h2 = .02, indicating that the effect of lexical content on vertical lip corner motion was mediated by emotion
These results confirm that lip corners differed across emotions, but did not differ across speech and song

Summary

Introduction

Throughout history, speech and song have served as overlapping and interchangeable forms of vocal expression. A significant body of research has focused on acoustic cues, identifying overlaps in the expression of emotion between speech and song (Cowie et al, 2001; Gabrielsson & Lindström, 2001; Ilie & Thompson, 2006; Juslin, 2001; Juslin & Laukka, 2003; Scherer, 1995, 2003). This emphasis on the acoustic modality, The authors thank Erik Koopmans, Frances Spidle, Rachel Brown, and Pascale Lidji of the Sequence Production Lab for their comments. Relatively little is known about the role of facial expressions in vocal performance (Livingstone, Thompson, & Russo, 2009; Thompson, Russo, & Quinto, 2008)

Methods

Results

Discussion

Conclusion