Vocal Emotion Research Articles

The Impact of Musical Rehabilitation on Complex Sound Perception in Cochlear Implant Users: A Systematic Review.

Musical rehabilitation has been used in clinical and nonclinical contexts to improve postimplantation auditory processing in implanted individuals. This systematic review aimed to evaluate the efficacy of music rehabilitation in controlled experimental and quasi-experimental studies on cochlear implant (CI) user speech and music perception. PubMed/MEDLINE, EMBASE, Web of Science, PsycARTICLES, and PsycINFO databases through July 2022. Controlled experimental trials and prospective studies were included if they compared pretest and posttest data and excluded hearing aid-only users. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were then used to extract data from 11 included studies with a total of 206 pediatric and adult participants. Interventions included group music therapy, melodic contour identification training, auditory-motor instruction, or structured digital music training. Studies used heterogeneous outcome measures evaluating speech and music perception. Risk of bias was assessed using the National Heart, Lung, and Blood Institute Quality Assessment Tool. A total of 735 studies were screened, and 11 met the inclusion criteria. Six trials reported both speech and music outcomes, whereas five reported only music perception outcomes after the intervention relative to control. For music perception outcomes, significant findings included improvements in melodic contour identification (five studies, p < 0.05), timbre recognition (three studies, p < 0.05), and song appraisal (three studies, p < 0.05) in their respective trials. For speech prosody outcomes, only vocal emotion identification demonstrated significant improvements (two studies, p < 0.05). Music rehabilitation improves performance on multiple measures of music perception, as well as tone-based characteristics of speech (i.e., emotional prosody). This suggests that rehabilitation may facilitate improvements in the discrimination of spectrally complex signals.

Roles of bilingualism and musicianship in resisting semantic or prosodic interference while recognizing emotion in sentences

Abstract Listeners can use the way people speak (prosody) or what people say (semantics) to infer vocal emotions. It can be speculated that bilinguals and musicians can better use the former rather than the latter compared to monolinguals and non-musicians. However, the literature to date has offered mixed evidence for this prosodic bias. Bilinguals and musicians are also arguably known for their ability to ignore distractors and can outperform monolinguals and non-musicians when prosodic and semantic cues conflict. In two online experiments, 1041 young adults listened to sentences with either matching or mismatching semantic and prosodic cues to emotions. 526 participants were asked to identify the emotion using the prosody and 515 using the semantics. In both experiments, performance suffered when cues conflicted, and in such conflicts, musicians outperformed non-musicians among bilinguals, but not among monolinguals. This finding supports an increased ability of bilingual musicians to inhibit irrelevant information in speech.

Evaluating the Relative Perceptual Salience of Linguistic and Emotional Prosody in Quiet and Noisy Contexts.

How people recognize linguistic and emotional prosody in different listening conditions is essential for understanding the complex interplay between social context, cognition, and communication. The perception of both lexical tones and emotional prosody depends on prosodic features including pitch, intensity, duration, and voice quality. However, it is unclear which aspect of prosody is perceptually more salient and resistant to noise. This study aimed to investigate the relative perceptual salience of emotional prosody and lexical tone recognition in quiet and in the presence of multi-talker babble noise. Forty young adults randomly sampled from a pool of native Mandarin Chinese with normal hearing listened to monosyllables either with or without background babble noise and completed two identification tasks, one for emotion recognition and the other for lexical tone recognition. Accuracy and speed were recorded and analyzed using generalized linear mixed-effects models. Compared with emotional prosody, lexical tones were more perceptually salient in multi-talker babble noise. Native Mandarin Chinese participants identified lexical tones more accurately and quickly than vocal emotions at the same signal-to-noise ratio. Acoustic and cognitive dissimilarities between linguistic prosody and emotional prosody may have led to the phenomenon, which calls for further explorations into the underlying psychobiological and neurophysiological mechanisms.

Vocal emotion recognition in attention-deficit hyperactivity disorder: a meta-analysis

ABSTRACT There is debate within the literature as to whether emotion dysregulation (ED) in Attention-Deficit Hyperactivity Disorder (ADHD) reflects deviant attentional mechanisms or atypical perceptual emotion processing. Previous reviews have reliably examined the nature of facial, but not vocal, emotion recognition accuracy in ADHD. The present meta-analysis quantified vocal emotion recognition (VER) accuracy scores in ADHD and controls using robust variance estimation, gathered from 21 published and unpublished papers. Additional moderator analyses were carried out to determine whether the nature of VER accuracy in ADHD varied depending on emotion type. Findings revealed a medium effect size for the presence of VER deficits in ADHD, and moderator analyses showed VER accuracy in ADHD did not differ due to emotion type. These results support the theories which implicate the role of attentional mechanisms in driving VER deficits in ADHD. However, there is insufficient data within the behavioural VER literature to support the presence of emotion processing atypicalities in ADHD. Future neuro-imaging research could explore the interaction between attention and emotion processing in ADHD, taking into consideration ADHD subtypes and comorbidities.

Event-related potentials indicate differential neural reactivity to species and valence information in vocal stimuli in sleeping dogs

Dogs live in a complex social environment where they regularly interact with conspecific and heterospecific partners. Awake dogs are able to process a variety of information based on vocalisations emitted by dogs and humans. Whether dogs are also able to process such information while asleep, is unknown. In the current explorative study, we investigated in N = 13 family dogs, neural response to conspecific and human emotional vocalisations. Data were recorded while dogs were asleep, using a fully non-invasive event-related potential (ERP) paradigm. A species (between 250–450 and 600–800 ms after stimulus onset) and a species by valence interaction (between 550 to 650 ms after stimulus onset) effect was observed during drowsiness. A valence (750–850 ms after stimulus onset) and a species x valence interaction (between 200 to 300 ms and 450 to 650 ms after stimulus onset) effect was also observed during non-REM specific at the Cz electrode. Although further research is needed, these results not only suggest that dogs neurally differentiate between differently valenced con- and heterospecific vocalisations, but they also provide the first evidence of complex vocal processing during sleep in dogs. Assessment and detection of ERPs during sleep in dogs appear feasible.

Automatic Brain Categorization of Discrete Auditory Emotion Expressions.

Seamlessly extracting emotional information from voices is crucial for efficient interpersonal communication. However, it remains unclear how the brain categorizes vocal expressions of emotion beyond the processing of their acoustic features. In our study, we developed a new approach combining electroencephalographic recordings (EEG) in humans with a frequency-tagging paradigm to 'tag' automatic neural responses to specificcategories of emotion expressions. Participants were presented with a periodic stream of heterogeneous non-verbal emotional vocalizations belonging to five emotion categories: anger, disgust, fear, happiness and sadness at 2.5Hz (stimuli length of 350ms with a 50ms silent gap between stimuli). Importantly, unknown to the participant, a specific emotion category appeared at a target presentation rate of 0.83Hz that would elicit an additional response in the EEG spectrum only if the brain discriminates the target emotion category from other emotion categories and generalizes across heterogeneous exemplars of the target emotion category. Stimuli were matched across emotion categories for harmonicity-to-noise ratio, spectral center of gravity and pitch. Additionally, participants were presented with a scrambled version of the stimuli with identical spectral content and periodicity but disrupted intelligibility. Both types of sequences had comparable envelopes and early auditory peripheral processing computed via the simulation of the cochlear response. We observed that in addition to the responses at the general presentation frequency (2.5Hz) in both intact and scrambled sequences, a greater peak in the EEG spectrum at the target emotion presentation rate (0.83Hz) and its harmonics emerged in the intact sequence in comparison to the scrambled sequence. The greater response at the target frequency in the intact sequence, together with our stimuli matching procedure, suggest that the categorical brain response elicited by a specific emotion is at least partially independent from the low-level acoustic features of the sounds. Moreover, responses at the fearful and happy vocalizations presentation rates elicited different topographies and differenttemporal dynamics, suggesting that different discrete emotions are represented differently in the brain. Our paradigm revealed the brain's ability to automatically categorize non-verbal vocal emotion expressions objectively (at a predefined frequency of interest), behavior-free, rapidly (in few minutes of recording time) and robustly (with a high signal-to-noise ratio), making it a useful tool to study vocal emotion processing and auditory categorization in generaland in populations where behavioral assessments are more challenging.

The Levels of Auditory Processing during Emotional Perception in Children with Autism.

The perception of basic emotional sounds, such as crying and laughter is associated with effective interpersonal communication. Difficulties with the perception and analysis of sounds that complicate understanding emotions at an early development age may contribute to communication deficits. This study focused on auditory nonverbal emotional perception including emotional vocalizations with opposite valences (crying and laughter) and neutral sound (phoneme "Pᴂ"). We conducted event-related potential analysis and compared peak alpha frequencies (PAFs) for different conditions in children with autism spectrum disorder (ASD) and typically developing (TD) children aged 4 to 6 years old (N = 25 for each group). Children with ASD had a higher amplitude of P100 and lower amplitude of N200 for all types of sounds and higher P270 in response to neutral phoneme. During the perception of emotional sounds, children with ASD demonstrated a single P270 electroencephalography (EEG) component instead of a P200-P300 complex specific to TD children. However, the most significant differences were associated with a response to emotional valences of stimuli. The EEG differences between crying and laughter were expressed as a lower amplitude of N400 and higher PAF for crying compared to laughter and were found only in TD children. Children with ASD have shown not just abnormal acoustical perception but altered emotional analysis of affective sounds as well.

Humans recognize affective cues in primate vocalizations: acoustic and phylogenetic perspectives

Humans are adept at extracting affective information from vocalizations of humans and other animals. However, the extent to which human recognition of vocal affective cues of other species is due to cross-taxa similarities in acoustic parameters or the phylogenetic closeness between species is currently unclear. To address this, we first analyzed acoustic variation in 96 affective vocalizations, taken from agonistic and affiliative contexts, of humans and three other primates—rhesus macaques (Macaca mulatta), chimpanzees and bonobos (Pan troglodytes and Pan paniscus). Acoustic analyses revealed that agonistic chimpanzee and bonobo vocalizations were similarly distant from agonistic human voices, but chimpanzee affiliative vocalizations were significantly closer to human affiliative vocalizations, than those of bonobos, indicating a potential derived vocal evolution in the bonobo lineage. Second, we asked 68 human participants to categorize and also discriminate vocalizations based on their presumed affective content. Results showed that participants reliably categorized human and chimpanzee vocalizations according to affective content, but not bonobo threat vocalizations nor any macaque vocalizations. Participants discriminated all species calls above chance level except for threat calls by bonobos and macaques. Our results highlight the importance of both phylogenetic and acoustic parameter level explanations in cross-species affective perception, drawing a more complex picture to the origin of vocal emotions.

Vocal Emotional Expression in Parkinson’s Disease: Roles of Sex and Emotions

Introduction: Parkinson’s disease (PD) commonly causes speech impairments, including difficulties in expressing emotions through voice. Method: The objective of this study was to investigate gendered vocal expressions of fear, anger, sadness, and happiness for mild to moderate PD. Prosodic features (related to fundamental frequency (F0), intensity (I), speech rate, articulation rate, and number and duration of pauses) and acoustic correlates of voice quality (CPPS, jitter, shimmer, and HNR) were collected from 14 patients with PD (mean age = 69.93; SD = 7.12; 8 males, 6 females) and 13 healthy controls (HC) (mean age = 68.13; SD = 8.27; 5 males, 8 females) matched for age, sex, and years of education. The utterances were extracted from four emotional and one neutral text. The neutral utterance and the emotional utterances were compared. Intra-sex comparison (female with PD vs. female HC and male with PD vs. male HC) and inter-sex comparison (female vs. male both for patients with PD and for HC), were performed with the Mann–Whitney test. A Mann–Whitney test was also used to compare the different emotional conditions, considering sex and PD diagnosis as well. Results: No significant intra-sex differences were found for the neutral speech, but inter-sex differences emerged. Regarding emotional speech, females with PD featured lower MaxF0 than female HCs for happiness and higher intensity variability (SD I) for sadness. Utterances by females with PD had lower CPPS than utterances by HCs for anger and fear. Utterances by males with PD had lower minimum intensity (MinI) than utterances by male HCs when expressing fear. Conclusions: Emotional vocal expression in individuals with PD was found to be impaired and showed sex differences. These findings have the potential to significantly impact the quality of life of PD patients.

Positive Valence Contributes to Hyperarticulation in Maternal Speech to Infants and Puppies.

Infant-directed speech often has hyperarticulated features, such as point vowels whose formants are further apart than in adult-directed speech. This increased "vowel space" may reflect the caretaker's effort to speak more clearly to infants, thus benefiting language processing. However, hyperarticulation may also result from more positive valence (e.g., speaking with positive vocal emotion) often found in mothers' speech to infants. This study was designed to replicate others who have found hyperarticulation in maternal speech to their 6-month-olds, but also to examine their speech to a non-human infant (i.e., a puppy). We rated both kinds of maternal speech for their emotional valence and recorded mothers' speech to a human adult. We found that mothers produced more positively valenced utterances and some hyperarticulation in both their infant- and puppy-directed speech, compared to their adult-directed speech. This finding promotes looking at maternal speech from a multi-faceted perspective that includes emotional state.

ECPEC: Emotion-Cause Pair Extraction in Conversations

Conversational sentiment analysis (CSA) and emotion-cause pair extraction (ECPE) tasks have attracted increasing attention in recent years. The former aims to predict the sentiment states of speakers in a conversation, and the latter is about extracting emotion-cause clauses in a document. However, one drawback of CSA is that it cannot model the causal reasoning among emotion and neutral utterances from different speakers. In this work, we propose a new task: emotion-cause pair extraction in conversations (ECPEC), which aims to extract pairs of emotional utterances and corresponding cause utterances in conversations. The utterance-level ECPEC task is more challenging since the distance between emotion and cause utterances is larger than that of the clause-level ECPE task. To this end, we build a novel dataset ConvECPE and propose a specifically designed two-step framework for the new ECPEC task. Experimental results on ConvECPE dataset demonstrate the feasibility of the ECPEC task as well as the effectiveness of our framework.

"Rear bias" in spatial auditory perception: Attentional and affective vigilance to sounds occurring outside the visual field.

Presented studies explored the rear bias phenomenon, that is, the attentional and affective bias to sounds occurring behind the listener. Physiological and psychological reactions (i.e., fEMG, EDA/SCR, Simple Reaction Task-SRT, and self-assessments of affect-related states) were measured in response to tones of different frequencies (Study 1) and emotional vocalizations (Study 2) presented in rear and front spatial locations. Results showed that emotional vocalizations, when located in the back, facilitate reactions related to attention orientation (i.e., auricularis muscle response and simple reaction times) and evoke higher arousal-both physiological (as measured by SCR) and psychological (self-assessment scale). Importantly, observed asymmetries were larger for negative and threat-related signals (e.g., anger) than positive/nonthreatening ones (e.g., achievement). By contrast, there were only small differences for the relatively higher frequency tones. The observed relationships are discussed in terms of one of the postulated auditory system's functions, which is monitoring of the environment in order to quickly detect potential threats that occur outside of the visual field (e.g., behind one's back).

Cross-Cultural Calibration of Words and Emotions: Referential, Constructionist, and Pragmatic Perspectives

Emotion-related words differ across societies and eras. Does this mean that emotions themselves differ in similar ways? Three perspectives on language-emotion relations suggest alternative answers to this question. A referential approach implies that any language's emotion concepts provide a potentially perfectible mapping of the emotional world. Constructionist approaches suggest that linguistic concepts shape culturally different emotion perceptions. By contrast, a pragmatic approach emphasizes the performative functions served by conversational uses of emotion words. From this perspective, emotional language is attuned to culture-specific requirements for aligning relations between people and objects. Thus, emotional utterances may be constituents of socially functional emotions rather than separate commentaries on them. Full understanding of cultural variation requires investigation of naturalistic emotional conversations in different societies.

Facial and Vocal Emotion Recognition in Adolescence: A Systematic Review

AbstractThe ability to recognize emotion is important to wellbeing and building relationships with others, making this skill important in adolescence. Research investigating adolescents’ ability to recognize facial and vocal emotion expressions has reported differing conclusions about the pattern of emotion recognition across this developmental period. This systematic review aimed to clarify the pattern of recognition for facial and vocal emotion expressions, and the relationship of performance to different task and emotion expression characteristics. A comprehensive and systematic search of the literature was conducted using six databases. To be eligible for inclusion, studies had to report data for adolescents between 11 and 18 years of age and measure accuracy of the recognition of emotion cues in either the face or voice. A total of 2333 studies were identified and 47 met inclusion criteria. The majority of studies focused on facial emotion recognition. Overall, early, mid-, and late-adolescents showed a similar pattern of recognition for both facial and vocal emotion expressions with the exception of Sad facial expressions. Sex of the participant also had minimal impact on the overall recognition of different emotions. However, analysis showed considerable variability according to task and emotion expression characteristics. Future research needs to increase focus on recognition of complex emotions, and low-intensity emotion expressions as well as the influence of the inclusion of Neutral as a response option.

Integration of auditory cues for vocal emotion perception – differences between musicians and non-musicians

Integration of auditory cues for vocal emotion perception – differences between musicians and non-musicians

An ontogenetic perspective of vocal emotion recognition

An ontogenetic perspective of vocal emotion recognition

Neural networks of vocal emotion production mechanisms and correlates of bodily vibrations

Neural networks of vocal emotion production mechanisms and correlates of bodily vibrations

Analysis of the Historical Origin of Ancient Chinese Poetry and Art Songs

The successful combination of poetry and dynamic beauty in poetry and dynamic beauty in music often brings vitality to people with transcendent aesthetic enjoyment. Classical poetry is an important cultural treasure in ancient Chinese literature, and it also has a relatively broad cultural value in the context of the current era, and is a brilliant pearl of Chinese culture. Ancient Chinese poetry and art songs refer to a relatively unique form of music genre produced by taking the poetry of Chinese literati and scholars as the singing content, adding scores and soundtracks by composers, and skillfully integrating poetry, rhyme and vocal emotion. After thousands of years of development, Chinese modern art songs combine literary art and musical art, precipitating thousands of years of cultural connotations of the Chinese nation; It embodies the spiritual feelings of thousands of people and is an important part of the excellent traditional culture of the Chinese nation. It is not only a perfect combination of traditional culture and elegant art, but also the symbiotic link between poetry and music. It has been spread for thousands of years in the long river of literature and art of the Chinese nation for a long time, and has profound historical connotation and high research value.

Cortical haemodynamic responses predict individual ability to recognise vocal emotions with uninformative pitch cues but do not distinguish different emotions.

We investigated the cortical representation of emotional prosody in normal-hearing listeners using functional near-infrared spectroscopy (fNIRS) and behavioural assessments. Consistent with previous reports, listeners relied most heavily on F0 cues when recognizing emotion cues; performance was relatively poor-and highly variable between listeners-when only intensity and speech-rate cues were available. Using fNIRS to image cortical activity to speech utterances containing natural and reduced prosodic cues, we found right superior temporal gyrus (STG) to be most sensitive to emotional prosody, but no emotion-specific cortical activations, suggesting that while fNIRS might be suited to investigating cortical mechanisms supporting speech processing it is less suited to investigating cortical haemodynamic responses to individual vocal emotions. Manipulating emotional speech to render F0 cues less informative, we found the amplitude of the haemodynamic response in right STG to be significantly correlated with listeners' abilities to recognise vocal emotions with uninformative F0 cues. Specifically, listeners more able to assign emotions to speech with degraded F0 cues showed lower haemodynamic responses to these degraded signals. This suggests a potential objective measure of behavioural sensitivity to vocal emotions that might benefit neurodiverse populations less sensitive to emotional prosody or hearing-impaired listeners, many of whom rely on listening technologies such as hearing aids and cochlear implants-neither of which restore, and often further degrade, the F0 cues essential to parsing emotional prosody conveyed in speech.

Perceived naturalness of emotional voice morphs

ABSTRACT Research into voice perception benefits from manipulation software to gain experimental control over acoustic expression of social signals such as vocal emotions. Today, parameter-specific voice morphing allows a precise control of the emotional quality expressed by single vocal parameters, such as fundamental frequency (F0) and timbre. However, potential side effects, in particular reduced naturalness, could limit ecological validity of speech stimuli. To address this for the domain of emotion perception, we collected ratings of perceived naturalness and emotionality on voice morphs expressing different emotions either through F0 or Timbre only. In two experiments, we compared two different morphing approaches, using either neutral voices or emotional averages as emotionally non-informative reference stimuli. As expected, parameter-specific voice morphing reduced perceived naturalness. However, perceived naturalness of F0 and Timbre morphs were comparable with averaged emotions as reference, potentially making this approach more suitable for future research. Crucially, there was no relationship between ratings of emotionality and naturalness, suggesting that the perception of emotion was not substantially affected by a reduction of voice naturalness. We hold that while these findings advocate parameter-specific voice morphing as a suitable tool for research on vocal emotion perception, great care should be taken in producing ecologically valid stimuli.