Emotional Cues Research Articles

Speech Emotion Recognition Using Feedforward Neural Network

Speech Emotion Recognition (SER) has gained increasing attention due to its application in fields such as human-computer interaction, healthcare, customer service automation, and affective computing. This review focuses on the application of Deep Neural Networks (DNNs), specifically Feedforward Neural Networks (FNNs), in detecting and classifying emotions from speech signals. Recent advancements in deep learning have significantly enhanced the ability of machines to interpret emotional cues from auditory data. This paper discusses the motivation, objectives, and scope of employing DNN FNN architectures for SER. We also address the technical feasibility and potential of this approach for real-time applications. By reviewing existing literature, this study aims to provide insights into the current progress, challenges, and future prospects of SER systems.

Remembering the blues: negative emotion during encoding improve memory recall in major depressive Disorder

ABSTRACT Substantial research indicates that individuals with Major Depressive Disorder (MDD) remember more negative information compared to neutral and positive information. This phenomenon is commonly attributed to attentional biases toward negative over neutral and positive information. A recent attentional resources model suggests that in MDD, negative cues not only capture attention, but also lead to deeper processing of subsequent information, irrespective of its content. This study aimed to replicate findings supporting this attentional resources model and go beyond it by investigating the effect of negative cues on encoding and retrieval processes. Forty-one participants with MDD and no comorbid diagnoses, and 42 healthy-controls completed the emotional recall task with negative or positive videos presented during encoding and retrieval stages of a neutral word-list memory test. During encoding, only the MDD group exhibited a difference between negative and positive videos, such that for negative videos memory recall was improved and for positive words it was reduced. Emotional videos had no effect when presented during retrieval. These results suggest that in MDD, encountering emotional cues not only biases retrieval processes toward recalling more negative content, but rather fundamentally alters the depth of information processing, while not leading to a broad-spectrum recruitment of cognitive resources.

Motivation, Learning Styles and Strategies in English Language among First-Year College Students in Public HEI’s in Sulu

English, as the most frequently spoken second language, and sometimes referred to as the global lingua franca, acts as a bridge for communication across cultures and nations. Learning English is a strong tool that allows people to fully participate in the global society by encouraging understanding, collaboration, and personal growth. This descriptive-comparative paper analyzed the motivation, learning styles, and learning strategies of first year college students from public higher education institutions (HEIs) in Sulu, Philippines. Participants were purposively sampled (n=200) to participate in the study. Findings indicated that first year college students in the province were integratively motivated to learn the English language. They particularly engaged in psychological and sociological learning styles through speaking to fluent English speakers, understanding emotional cues, reflection, and self-assessment. They engage in metacognitive (planning, organizing thoughts) and cognitive (assessment, analysis) to learn and understand English. Educators should adopt a holistic approach to teaching English, one that motivates students, respects their individual learning styles, and equips them with effective learning strategies. Such an approach could likely lead to more effective and meaningful language learning experiences, which could improve students' proficiency and confidence in using English.

Contemporary music improvisation for silent film

This article explores how contemporary music improvisation for silent film both evokes and challenges the jazz canon by pushing the boundaries of genre, instrumentation, techniques, and collaboration while also engaging with the narrative and cultural contexts of the films. It draws from my practice-led research over the last twenty years, which has consisted of performing free improvised music in response to the screening of silent film, as a solo performer or as part of a jazz ensemble. The corpus of films used in the research spans animated shorts by Dutch director Gerrit van Dijk, surrealist shorts by Man Ray, German expressionist feature film by Fritz Lang and F. W. Murnau, French feminist film by Germaine Dulac, and documentary and amateur archive film of the North West Film Archive in Manchester and the Media Archive for Central England in Lincoln. In the performances, while jazz is taken as primary influence, musicians often incorporate elements of other musics, make use of a broader range of instruments and of non-traditional sound sources beyond the typical jazz ensemble, and employ experimental techniques and unconventional playing methods that challenge and evoke the conventional improvisational norms of jazz. Musicians might opt to follow or contradict the narrative and emotional cues of the film, which can result in processes of music making that prioritize the exploration of the boundaries of storytelling over traditional jazz forms and structures. Across this research, I have observed how silent film improvisation has been attracting diverse audiences. This cross-pollination of audiences can introduce jazz to new fans and challenge preconceived notions about who jazz is for.

How “Light” Is “Light Smoking”? On the Cognitive Power of Nicotine Dependence

In recent years, habits related to smoking have been changing. An increasing portion of light/occasional smokers tend to define themselves as non-smokers, leading to an incorrect perception of the risks that smoking even a few cigarettes can entail. In this study, we investigated the nicotine-induced cognitive distortion in young, higher-education students with low/moderate dependence (as indexed by the Fagerstrom questionnaire). The study involved 111 participants (62 female; mean age 24.43 ± 3.77) divided into smokers and non-smokers, who responded to specific questionnaires to evaluate their attachment style, emotion dysregulation, and state anxiety. Their response to smoking-related cues following emotional stimulation was experimentally evaluated, with participants being made to choose between care- or smoking-related images, following the presentation of threatening or neutral stimuli. The results show a cognitive bias in smokers, with participants choosing smoking-related stimuli significantly more often than non-smokers, with a slower reaction time, regardless of emotional cues. Emotion dysregulation and attachment style were also significantly correlated with response choice but not with response latency. Overall, our data indicate that there is no such thing as light use of nicotine and that smoking, even if not continuous, determines cognitive biases that lead to a vision of the environment as a function of substance seeking.

Optimization Research on Integrating Mental Health Into Ideological and Political Theory Courses Through Deep Learning Network Applications

This study aims to evaluate the effectiveness of incorporating mental health education into ideological and political theory classrooms using speech emotion recognition technology from the field of human-computer interaction. Mental health education is essential for students' well-being, and this study establishes a theoretical framework for its integration into the curriculum. Utilizing recurrent neural networks (RNNs) augmented with attention mechanisms, the research demonstrates the model's proficiency in identifying subtle emotional cues within speech data. The model achieves an average recognition accuracy of 87.21% on the RAVDESS speech emotion corpus, showing particular strengths in detecting emotions like happiness and boredom. The findings suggest that this technology can be effectively employed to gauge student satisfaction with different teaching methodologies post-integration, offering valuable insights for refining educational practices in ideological and political theory classes through improved mental health education initiatives.

Deep Learning Based Song Recommendation System Using Facial Expression And Heart Rate

In today’s music streaming landscape, personalized song recommendations are crucial for enhancing user experiences. This research study combines heart rate monitoring and facial expression detection to present a novel deep learning method for song suggestion. With a 92% accuracy rate, a Convolutional Neural Network (CNN) model is used to identify emotional states from facial expressions, and heart rate information adds more context to the user's emotional intensity. The large music library is matched to these emotional cues, guaranteeing that suggested songs correspond with the user's present emotional state. This approach provides a really interesting, emotionally stirring, and contextually appropriate music finding experience.

Constrained Emotional Sentence Production in Parkinson's Disease.

Deficits in the processing and production of emotional cues are well documented in the Parkinson's disease (PD) literature; however, few have ventured to explore how impairments may impact emotional language use in this population, particularly beyond the word level. Emotional language is an important multidimensional manner of communicating one's wants and needs; thus, the current study sought to explore how various aspects of language production may be impacted by the emotionality of a stimulus. Eighteen persons with PD and 22 healthy adults completed a constrained emotional sentence production task in which the affective target word was either a noun or a verb. Output was analyzed for fluency, grammaticality, completeness, and response initiation times. Cognitive (i.e., working memory [WM], inhibition, and switching) and mood (i.e., depression and apathy) measures were examined as factors influencing performance. Individuals with PD produced fewer fluent responses than healthy controls. Furthermore, they had fewer grammatical responses in their production of negative sentences and exhibited reduced information completeness when producing sentences containing positive stimuli. Group differences could not be wholly attributed to individual differences in WM or apathy. Our results support those of others that document language production deficits in individuals with PD above and beyond those impairments that can be explained by the select cognitive abilities explored here. Moreover, the emotionality of the topic may impact various aspects of communicative competence in persons with PD. For instance, disease processes associated with degeneration of neural substrates important for processing negative stimuli may also impact the grammaticality of productions containing negatively valenced content. Thus, it is important to consider how individuals in this population communicate during emotional circumstances. https://doi.org/10.23641/asha.27289413.

Category-dependent contribution of dog facial and bodily cues in human perception of dog emotions

The increasing popularity of human-dog interaction in our society calls for fuller understanding of humans’ ability to appraise dogs’ affective states, yet most research only focuses on recognizing dog facial expressions of primary/basic emotions. While the face is the dominant human emotional expression channel, bodily cues are also informative indicators of dog emotional states. In this online study, with dynamic and naturalistic videos depicting a common range of dog primary (anger, disgust, fear, happiness, sadness, surprise) and secondary emotions (appeasement, frustration, pain, positive anticipation, separation-distress), we compared human performance from 447 participants in categorizing dog facial expressions (with a visible dog face only) and bodily expressions (with a visible dog face and body). The analysis revealed when averaging across all tested emotions, bodily expression tended to attract higher categorization accuracy than facial expression. However, the two expression channels demonstrated category-dependent modification of dog emotion categorization accuracy (e.g., higher accuracy in recognizing facial expressions of anger and surprise, but bodily expressions of happiness and fear) and bias (e.g., mistaking fear facial expression as happiness, but fear bodily expression as sadness). Furthermore, the impact of owner experience on recognizing dog emotions was also modulated by the expression channel and emotion category (e.g., prolonged experience with dogs tended to improve performance in recognizing the fear facial expression, and the appeasement bodily expression). Taken together, these results suggest that different channels of emotional expression by dogs may transmit category-specific diagnostic emotional cues, which aid human appraisal of their affective states.

Cross-Modal Harmony: Low-Rank Fusion for Enhanced Artificial Emotion Recognition

Emotion recognition has become a subject of considerable interest in recent times, owing to its diverse and far-reaching applications in various fields. These applications span from enhancing human-computer interactions to assessing mental health and improving entertainment systems. The proposed study presents a novel approach for emotion recognition by fusing audio and video modalities using low-rank fusion techniques. The proposed methodology leverages the complementary nature of audio and video data in capturing emotional cues. Audio data often encapsulates tone, speech patterns, and vocal nuances, while video data captures facial expressions, body language, and gestures. However, the challenge lies in effectively integrating these two modalities to enhance recognition accuracy. To address the challenge, it employs low-rank fusion, a dimensionality reduction technique that extracts the most informative features from both modalities while minimizing redundancy. Furthermore, it presents the implementation of the chosen low-rank fusion algorithm in a real-world emotion recognition system. The results can contribute to advancing the field of emotion recognition by providing a practical and efficient solution for combining audio and video data to achieve more robust and accurate emotion classification. Keywords: Deep Learning; Emotion Recognition; Human-Computer Interaction; Low-Rank Fusion; Multimodal Fusion.

Changes in P300 amplitude to negative emotional stimuli correlate with treatment responsiveness to sertraline in adolescents with depression

ObjectiveAdolescents with depression is characterized by high rates of recurrence and functional impairment, with a significant association with suicide risk. Antidepressants are commonly prescribed to treat depression, yet few reproducible neurobiological markers for depression and antidepressant treatment response have been identified. Therefore, discovering a stable and reliable neurobiological marker holds significant value for both the clinical diagnosis and treatment of depression in adolescents. MethodsOne hundred and seven patients with major depressive disorder (MDD group, 30 males, 77 females, mean age: 14.80 years), and 25 healthy subjects (HC group, 13 males, 12 females, mean age: 15.72 years) were recruited to perform a two-choice oddball task related to negative emotional cues. All participants completed a self-administered questionnaire to gather demographic information. A trained psychiatrist administered the Hamilton Depression Scale (HAMD-17) to assess depression severity. Of the 107 adolescents with depression, 61 received antidepressant medication for 8 weeks, and 61 of these patients were followed up. Multichannel EEG was recorded continuously from 64 scalp electrodes using the Curry 8 system. EEG signal preprocessing and analysis was performed offline using the EEGLAB toolbox in MATLAB. The ERP component characteristics associated with emotional processing were extracted from the difference waves and statistically analyzed. ResultsAdolescents with depression exhibited significantly larger P300 amplitudes than healthy controls in response to both neutral and negative emotional cues. Following sertraline treatment, both depression scores and P300 amplitudes decreased significantly in adolescents with depression. Moreover, a strong positive correlation was observed between changes in depression scores and changes in P300 amplitude in response to negative emotional cues before and after treatment. ConclusionsChanges in neural reactivity to negative emotional stimuli among adolescents with depression can be selectively modulated by sertraline and are significantly associated with improvements in depressive symptoms. SignificanceChanges in P300 amplitude to negative emotional stimuli significantly correlate with treatment responsiveness to sertraline in adolescents with depression.

ESERNet: Learning spectrogram structure relationship for effective speech emotion recognition with swin transformer in classroom discourse analysis

Speech emotion recognition (SER) has received increased attention due to its extensive applications in many fields, especially in the analysis of teacher-student dialogue in classroom environment. It can help teachers to better learn about students’ emotions and thereby adjust teaching activities. However, SER has faced several challenges, such as the intrinsic ambiguity of emotions and the complex task of interpreting emotions from speech in noisy environments. These issues can result in reduced recognition accuracy due to a focus on less relevant or insignificant features. To address these challenges, this paper presents ESERNet, a Transformer-based model designed to effectively extract crucial clues from speech data by capturing both pivotal cues and long-range relationships in speech signal. The major contribution of our approach is a two-pathway SER framework. By leveraging the Transformer architecture, ESERNet captures long-range dependencies within speech mel-spectrograms, enabling a refined understanding of the emotional cues embedded in speech signals. Extensive experiments were conducted on the IEMOCAP and EmoDB datasets, the results show that ESERNet achieves state-of-the-art performance in SER and outperforms existing methods by effectively leveraging critical clues and capturing long-range dependencies in speech data. These results highlight the effectiveness of the model in addressing the complex challenges associated with SER tasks.

Decline of Affective Prosody Recognition With a Positivity Bias Among Older Adults: A Systematic Review and Meta-Analysis.

Understanding how older adults perceive and interpret emotional cues in speech prosody contributes to our knowledge of cognitive aging. This study provides a systematic review with meta-analysis to investigate the extent of the decline in affective prosody recognition (APR) among older adults in terms of overall and emotion-specific performance and explore potential moderators that may cause between-studies heterogeneity. The literature search encompassed five electronic databases, with a specific emphasis on studies comparing the APR performance of older adults with that of younger adults. This comparison was focused on basic emotions. Meta-regression analyses were executed to pinpoint potential moderators related to demographic and methodological characteristics. A total of 19 studies were included in the meta-analysis, involving 560 older adults with a mean age of 69.15 years and 751 younger adults with a mean age of 23.02 years. The findings indicated a substantial negative effect size (g = -1.21). Furthermore, the magnitude of aggregated effect sizes showed a distinct valence-related recognition pattern with positive prosody exhibiting smaller effect sizes. Language background and years of education were found to moderate the overall and emotion-specific (i.e., disgust and surprise) performance effect estimate, and age and gender significantly influenced the effect estimate of happiness. The results confirmed a significant decline in APR ability among older adults compared to younger adults, but this decline was unbalanced across basic emotions. Language background and educational level emerged as significant factors influencing older adults' APR ability. Moreover, participants with a higher mean age exhibited notably poorer performance in recognizing happy prosody. These findings underscore the need to further investigate the neurobiological mechanisms for APR decline associated with aging. https://doi.org/10.23641/asha.26407888.

CIRG-SL: Commonsense Inductive Relation Graph framework with Soft Labels for Empathetic Response Generation

Empathy, the ability to understand and respond to others’ emotions, is critical in dialogue systems, particularly for applications such as psychological counselling and casual conversation. However, existing approaches often struggle to accurately capture emotional transitions in user utterances and dialogue contexts across multiple levels of granularity. Additionally, the orthogonality of one-hot emotion representations overlooks the subtle relationships between emotion categories. To overcome these limitations, we propose a novel framework: the commonsense inductive relation graph with fine emotional soft labels for empathetic response generation. This framework refines the representation of user utterances by incorporating commonsense knowledge to derive external emotional cues, which are integrated into an adaptive inductive relation graph. This graph explicitly captures and models dynamic emotional trajectories throughout the dialogue. Additionally, we propose a fine-grained soft label construction method that more accurately reflects the nuanced distribution of emotions, replacing the rigid one-hot encoding with a continuous representation that better captures inter-emotion relationships. Our approach achieves strong performance on the benchmark EmpatheticDialogues dataset, with a perplexity score of 27.85, Dist-1 of 1.81, Dist-2 of 6.58, and a dialogue emotion accuracy of 55.65%. These results highlight the effectiveness of our framework in capturing subtle emotions and generating contextually rich, empathetic responses.

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

The impact of emotional expression by artificial intelligence recommendation chatbots on perceived humanness and social interactivity

Artificial intelligence-powered chatbots capable of expressing emotions have gained significant popularity in the realm of customer service. Although previous studies have explored the impact of emotional expression in chatbots, there is a lack of understanding regarding the precise effects of different emotional cues. In this study, we drew upon social presence theory to investigate how different emotional cues conveyed by recommendation chatbots affect perceived humanness, social interactivity, and social presence. We conducted a series of scenario-based online experiments to shed light on these dynamics. We found that all three emotional cues (text, emoticons, and images) employed by chatbots can increase perceived humanness and social interactivity. Social presence appears to be an underlying mechanism for these positive relationships. We also observed two-way interactions for any pair of emotional cues and a three-way interaction for all three emotional cues. Ultimately, to elicit the most favorable customer perception, we propose that a mode of emotional expression using either text or emoticons alone is most appropriate. These findings deepen our understanding of the impact of emotional expressions in chatbots and offer novel insights into how to deploy chatbots in customer service.

Perceived prolonged stress leads to difficulties in recognizing sadness from voice cues in men but not women.

It has long been known that stress has detrimental effects on cognition (e.g., Alderson & Novack, 2002; Lupien & Lepage, 2001), most notably documented for memory functions (e.g., Schwabe & Wolf, 2013). Interestingly, less is known about the effects of stress on other cognitive functions including language processing. Here, we have examined the effects of self-reported prolonged stress on recognition of emotional language content with a particular emphasis on gender differences. We tested how well 399 participants with different perceived stress levels recognized emotional voice cues. Findings confirm previous results from the emotional prosody literature by demonstrating that women generally outperform men in the vocal emotion recognition task. Crucially, results also revealed that medium levels of perceived stress impair the ability to detect sadness from voice cues in men but not women. These findings were not modulated by task demands (e.g., speeded response) or better acoustic discrimination abilities in women. Results are in line with the idea that perceived stress has a different impact on men versus women and that women have a higher level of experience in voice sadness recognition, potentially due to their predominant role as primary caretakers. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

What Is Faster than Where in Vocal Emotional Perception.

Voices carry a vast amount of information about speakers (e.g., emotional state; spatial location). Neuroimaging studies postulate that spatial ("where") and emotional ("what") cues are processed by partially independent processing streams. Although behavioral evidence reveals interactions between emotion and space, the temporal dynamics of these processes in the brain and its modulation by attention remain unknown. We investigated whether and how spatial and emotional features interact during voice processing as a function of attention focus. Spatialized nonverbal vocalizations differing in valence (neutral, amusement, anger) were presented at different locations around the head, whereas listeners discriminated either the spatial location or emotional quality of the voice. Neural activity was measured with ERPs of the EEG. Affective ratings were collected at the end of the EEG session. Emotional vocalizations elicited decreased N1 but increased P2 and late positive potential amplitudes. Interactions of space and emotion occurred at the salience detection stage: neutral vocalizations presented at right (vs. left) locations elicited increased P2 amplitudes, but no such differences were observed for emotional vocalizations. When task instructions involved emotion categorization, the P2 was increased for vocalizations presented at front (vs. back) locations. Behaviorally, only valence and arousal ratings showed emotion-space interactions. These findings suggest that emotional representations are activated earlier than spatial representations in voice processing. The perceptual prioritization of emotional cues occurred irrespective of task instructions but was not paralleled by an augmented stimulus representation in space. These findings support the differential responding to emotional information by auditory processing pathways.

Modulatory effects of goal relevance on emotional attention reveal that fear has a distinct value

ABSTRACT Threat-related stimuli can capture attention. However, it remains debated whether this capture is automatic or not. To address this question, we compared attentional biases to emotional faces using a dot-probe task (DPT) where emotion was never goal-relevant (Experiment 1) or made directly task-relevant by means of induction trials (Experiments 2–3). Moreover, the contingency between the DPT and induction trials was either partial (Experiment 2) or full (Experiment 3). Eye-tracking was used to ascertain that the emotional cue and the subsequent target were processed with peripheral vision. Experiments 1 and 2 both showed that negative faces captured attention, with faster target processing when it appeared on the same side as the preceding fearful face (i.e. fear-valid trials) compared to the opposite side where the neutral face was shown (i.e. fear-invalid trials), but also when it appeared on the side of the preceding neutral face (i.e. happy-invalid trials) compared to the happy face (i.e. happy-valid trials). Importantly, this preferential spatial orienting to negative emotion was not observed in Experiment 3, where the goal relevance of emotion was high. However, in that experiment, fearful faces produced a specific attentional bias during the DPT, which was mostly driven by the induction trials themselves.

The association between social rewards and anxiety: Links from neurophysiological analysis in virtual reality and social interaction game

Individuals’ affective experience can be intricate, influenced by various factors including monetary rewards and social factors during social interaction. However, within this array of factors, divergent evidence has been considered as potential contributors to social anxiety. To gain a better understanding of the specific factors associated with anxiety during social interaction, we combined a social interaction task with neurophysiological recordings obtained through an anxiety-elicitation task conducted in a Virtual Reality (VR) environment. Employing inter-subject representational similarity analysis (ISRSA), we explored the potential linkage between individuals’ anxiety neural patterns and their affective experiences during social interaction. Our findings suggest that, after controlling for other factors, the influence of the partner's emotional cues on individuals’ affective experiences is specifically linked to their neural pattern of anxiety. This indicates that the emergence of anxiety during social interaction may be particularly associated with the emotional cues provided by the social partner, rather than individuals’ own reward or prediction errors during social interaction. These results provide further support for the cognitive theory of social anxiety and extend the application of VR in future cognitive and affective studies.