Perceived Emotional Expression in Synthesized Performances of a Short Melody: Capturing the Listener's Judgment Policy

Abstract

Recent studies of music performance have shown that systematic variations in tempo, sound level, articulation, and timbre may be used by the performer to communicate representations of specific emotions to listeners. However, although performance analyses show that performers use certain cues to communicate emotions, they cannot explain how listeners use these cues to decode the expression. The purpose of this study was thus to examine listeners' cue utilization. This was done in two listening experiments using synthesized performances of a short melody. In the first experiment, an attempt was made to recreate representative cue profiles of five emotional expressions; happiness, sadness, anger, fear, and tenderness. The cues manipulated included tempo, sound level, spectrum, articulation, attack, vibrato, and timing. Listeners made forced choice judgments regarding the intended emotional expression. The results showed that (a) listeners were successful in decoding the intended emotional expression of the synthesized performances, (b) decoding accuracy was as high for synthesized performances as for live performances, and (c) reversal of the sequences reduced decoding accuracy to a larger extent for live performances than for synthesized performances, suggesting that live performances were relatively more dependent on prosodic contours. In the second experiment, five cues – tempo, sound level, spectrum, articulation, and attack – were systematically varied in a factorial design. Listeners were instructed to describe the emotional expression of each cue combination by ratings on adjective scales. An attempt to describe the listener's judgment policy was made using multiple regression. The results showed that (a) listeners used all of the available cues in their judgments, (b) none of the cues had more than a probabilistic (i.e., uncertain) relation to the listeners' judgments, (c) the predictive strength of each cue varied a lot depending on the emotion judged, and (d) the hypotheses based on our earlier studies of emotional expression in music performance (e.g., Gabrielsson & Juslin, 1996; Juslin; 1997d) were supported.