Acoustic cues for vocal emotion recognition by normal-hearing listeners and cochlear implant users

Abstract

Our recent study found that cochlear implant (CI) users’ quality of life in auditory, psychological, and social functioning were predicted by vocal emotion rather than sentence recognition scores. To eventually improve vocal emotion recognition with CIs, this study investigated the acoustic cues for vocal emotion recognition by CI users with CI alone or bimodal fitting, as compared to normal-hearing (NH) listeners. Sentence duration, mean fundamental frequency (F0), and F0 range were individually normalized for emotional utterances of each talker and sentence. In two other conditions, emotional utterances were presented backward in time and with upside-down F0 contours, respectively. Perceptual results showed significant effects of subject group, cue condition, talker, and emotion. Time-reversed utterances worsened NH listeners’ recognition of all emotions except sad, while upside-down F0 contours worsened that of angry and happy. Vocal emotion recognition with CI alone only degraded with time-reversed utterances. Time-reversed utterances worsened bimodal CI users’ recognition of angry and neutral, while upside-down F0 contours worsened that of angry and happy. Bimodal CI users and NH listeners were also affected by mean F0 and F0 range normalization when recognizing happy. We conclude that natural F0 contours should be faithfully encoded with CIs for better vocal emotion recognition.