Acoustic divergence from the training sample determines talker identification accuracy for emotional voices

Abstract

Different emotional states introduce substantial acoustic variations in talkers’ voices. It remains unclear how within-talker variability across emotional states affects listeners' ability to maintain perceptual constancy during talker identification. Here, we investigated (1) how changes in talkers’ emotional state affected talker identification accuracy, (2) how emotional state affected key features of voice acoustics, and (3) how emotion-related changes in these acoustic features affected listeners’ talker identification performance. Forty-eight listeners learned to identify talkers from speech expressing one emotional state (neutral, fearful, or angry) and then attempted to generalize that knowledge to speech expressing another emotional state. Talker identification accuracy was significantly worse in untrained emotions. Changes in voice acoustics across emotions were characterized for mean F0, F0 variability, jitter, HNR, speaking rate, and mean F2. To determine how emotion-related acoustic changes affected talker identification, we modeled talker identification accuracy as a function of difference in these features between training and test stimuli. Accuracy decreased as acoustic differences increased, regardless of talkers’ emotion. Thus, perceptual constancy depends on acoustic similarity to prior experience with a talker’s voice. Larger acoustic deviations, like those introduced by changes in emotional state, are more likely to cause a listener to misidentify a talker.