Emotional Cues Research Articles

How “Light” Is “Light Smoking”? On the Cognitive Power of Nicotine Dependence

In recent years, habits related to smoking have been changing. An increasing portion of light/occasional smokers tend to define themselves as non-smokers, leading to an incorrect perception of the risks that smoking even a few cigarettes can entail. In this study, we investigated the nicotine-induced cognitive distortion in young, higher-education students with low/moderate dependence (as indexed by the Fagerstrom questionnaire). The study involved 111 participants (62 female; mean age 24.43 ± 3.77) divided into smokers and non-smokers, who responded to specific questionnaires to evaluate their attachment style, emotion dysregulation, and state anxiety. Their response to smoking-related cues following emotional stimulation was experimentally evaluated, with participants being made to choose between care- or smoking-related images, following the presentation of threatening or neutral stimuli. The results show a cognitive bias in smokers, with participants choosing smoking-related stimuli significantly more often than non-smokers, with a slower reaction time, regardless of emotional cues. Emotion dysregulation and attachment style were also significantly correlated with response choice but not with response latency. Overall, our data indicate that there is no such thing as light use of nicotine and that smoking, even if not continuous, determines cognitive biases that lead to a vision of the environment as a function of substance seeking.

Optimization Research on Integrating Mental Health Into Ideological and Political Theory Courses Through Deep Learning Network Applications

This study aims to evaluate the effectiveness of incorporating mental health education into ideological and political theory classrooms using speech emotion recognition technology from the field of human-computer interaction. Mental health education is essential for students' well-being, and this study establishes a theoretical framework for its integration into the curriculum. Utilizing recurrent neural networks (RNNs) augmented with attention mechanisms, the research demonstrates the model's proficiency in identifying subtle emotional cues within speech data. The model achieves an average recognition accuracy of 87.21% on the RAVDESS speech emotion corpus, showing particular strengths in detecting emotions like happiness and boredom. The findings suggest that this technology can be effectively employed to gauge student satisfaction with different teaching methodologies post-integration, offering valuable insights for refining educational practices in ideological and political theory classes through improved mental health education initiatives.

Deep Learning Based Song Recommendation System Using Facial Expression And Heart Rate

In today’s music streaming landscape, personalized song recommendations are crucial for enhancing user experiences. This research study combines heart rate monitoring and facial expression detection to present a novel deep learning method for song suggestion. With a 92% accuracy rate, a Convolutional Neural Network (CNN) model is used to identify emotional states from facial expressions, and heart rate information adds more context to the user's emotional intensity. The large music library is matched to these emotional cues, guaranteeing that suggested songs correspond with the user's present emotional state. This approach provides a really interesting, emotionally stirring, and contextually appropriate music finding experience.

Constrained Emotional Sentence Production in Parkinson's Disease.

Deficits in the processing and production of emotional cues are well documented in the Parkinson's disease (PD) literature; however, few have ventured to explore how impairments may impact emotional language use in this population, particularly beyond the word level. Emotional language is an important multidimensional manner of communicating one's wants and needs; thus, the current study sought to explore how various aspects of language production may be impacted by the emotionality of a stimulus. Eighteen persons with PD and 22 healthy adults completed a constrained emotional sentence production task in which the affective target word was either a noun or a verb. Output was analyzed for fluency, grammaticality, completeness, and response initiation times. Cognitive (i.e., working memory [WM], inhibition, and switching) and mood (i.e., depression and apathy) measures were examined as factors influencing performance. Individuals with PD produced fewer fluent responses than healthy controls. Furthermore, they had fewer grammatical responses in their production of negative sentences and exhibited reduced information completeness when producing sentences containing positive stimuli. Group differences could not be wholly attributed to individual differences in WM or apathy. Our results support those of others that document language production deficits in individuals with PD above and beyond those impairments that can be explained by the select cognitive abilities explored here. Moreover, the emotionality of the topic may impact various aspects of communicative competence in persons with PD. For instance, disease processes associated with degeneration of neural substrates important for processing negative stimuli may also impact the grammaticality of productions containing negatively valenced content. Thus, it is important to consider how individuals in this population communicate during emotional circumstances. https://doi.org/10.23641/asha.27289413.

Category-dependent contribution of dog facial and bodily cues in human perception of dog emotions

The increasing popularity of human-dog interaction in our society calls for fuller understanding of humans’ ability to appraise dogs’ affective states, yet most research only focuses on recognizing dog facial expressions of primary/basic emotions. While the face is the dominant human emotional expression channel, bodily cues are also informative indicators of dog emotional states. In this online study, with dynamic and naturalistic videos depicting a common range of dog primary (anger, disgust, fear, happiness, sadness, surprise) and secondary emotions (appeasement, frustration, pain, positive anticipation, separation-distress), we compared human performance from 447 participants in categorizing dog facial expressions (with a visible dog face only) and bodily expressions (with a visible dog face and body). The analysis revealed when averaging across all tested emotions, bodily expression tended to attract higher categorization accuracy than facial expression. However, the two expression channels demonstrated category-dependent modification of dog emotion categorization accuracy (e.g., higher accuracy in recognizing facial expressions of anger and surprise, but bodily expressions of happiness and fear) and bias (e.g., mistaking fear facial expression as happiness, but fear bodily expression as sadness). Furthermore, the impact of owner experience on recognizing dog emotions was also modulated by the expression channel and emotion category (e.g., prolonged experience with dogs tended to improve performance in recognizing the fear facial expression, and the appeasement bodily expression). Taken together, these results suggest that different channels of emotional expression by dogs may transmit category-specific diagnostic emotional cues, which aid human appraisal of their affective states.

Cross-Modal Harmony: Low-Rank Fusion for Enhanced Artificial Emotion Recognition

Emotion recognition has become a subject of considerable interest in recent times, owing to its diverse and far-reaching applications in various fields. These applications span from enhancing human-computer interactions to assessing mental health and improving entertainment systems. The proposed study presents a novel approach for emotion recognition by fusing audio and video modalities using low-rank fusion techniques. The proposed methodology leverages the complementary nature of audio and video data in capturing emotional cues. Audio data often encapsulates tone, speech patterns, and vocal nuances, while video data captures facial expressions, body language, and gestures. However, the challenge lies in effectively integrating these two modalities to enhance recognition accuracy. To address the challenge, it employs low-rank fusion, a dimensionality reduction technique that extracts the most informative features from both modalities while minimizing redundancy. Furthermore, it presents the implementation of the chosen low-rank fusion algorithm in a real-world emotion recognition system. The results can contribute to advancing the field of emotion recognition by providing a practical and efficient solution for combining audio and video data to achieve more robust and accurate emotion classification. Keywords: Deep Learning; Emotion Recognition; Human-Computer Interaction; Low-Rank Fusion; Multimodal Fusion.

Changes in P300 amplitude to negative emotional stimuli correlate with treatment responsiveness to sertraline in adolescents with depression

ObjectiveAdolescents with depression is characterized by high rates of recurrence and functional impairment, with a significant association with suicide risk. Antidepressants are commonly prescribed to treat depression, yet few reproducible neurobiological markers for depression and antidepressant treatment response have been identified. Therefore, discovering a stable and reliable neurobiological marker holds significant value for both the clinical diagnosis and treatment of depression in adolescents. MethodsOne hundred and seven patients with major depressive disorder (MDD group, 30 males, 77 females, mean age: 14.80 years), and 25 healthy subjects (HC group, 13 males, 12 females, mean age: 15.72 years) were recruited to perform a two-choice oddball task related to negative emotional cues. All participants completed a self-administered questionnaire to gather demographic information. A trained psychiatrist administered the Hamilton Depression Scale (HAMD-17) to assess depression severity. Of the 107 adolescents with depression, 61 received antidepressant medication for 8 weeks, and 61 of these patients were followed up. Multichannel EEG was recorded continuously from 64 scalp electrodes using the Curry 8 system. EEG signal preprocessing and analysis was performed offline using the EEGLAB toolbox in MATLAB. The ERP component characteristics associated with emotional processing were extracted from the difference waves and statistically analyzed. ResultsAdolescents with depression exhibited significantly larger P300 amplitudes than healthy controls in response to both neutral and negative emotional cues. Following sertraline treatment, both depression scores and P300 amplitudes decreased significantly in adolescents with depression. Moreover, a strong positive correlation was observed between changes in depression scores and changes in P300 amplitude in response to negative emotional cues before and after treatment. ConclusionsChanges in neural reactivity to negative emotional stimuli among adolescents with depression can be selectively modulated by sertraline and are significantly associated with improvements in depressive symptoms. SignificanceChanges in P300 amplitude to negative emotional stimuli significantly correlate with treatment responsiveness to sertraline in adolescents with depression.

ESERNet: Learning spectrogram structure relationship for effective speech emotion recognition with swin transformer in classroom discourse analysis

Speech emotion recognition (SER) has received increased attention due to its extensive applications in many fields, especially in the analysis of teacher-student dialogue in classroom environment. It can help teachers to better learn about students’ emotions and thereby adjust teaching activities. However, SER has faced several challenges, such as the intrinsic ambiguity of emotions and the complex task of interpreting emotions from speech in noisy environments. These issues can result in reduced recognition accuracy due to a focus on less relevant or insignificant features. To address these challenges, this paper presents ESERNet, a Transformer-based model designed to effectively extract crucial clues from speech data by capturing both pivotal cues and long-range relationships in speech signal. The major contribution of our approach is a two-pathway SER framework. By leveraging the Transformer architecture, ESERNet captures long-range dependencies within speech mel-spectrograms, enabling a refined understanding of the emotional cues embedded in speech signals. Extensive experiments were conducted on the IEMOCAP and EmoDB datasets, the results show that ESERNet achieves state-of-the-art performance in SER and outperforms existing methods by effectively leveraging critical clues and capturing long-range dependencies in speech data. These results highlight the effectiveness of the model in addressing the complex challenges associated with SER tasks.

Decline of Affective Prosody Recognition With a Positivity Bias Among Older Adults: A Systematic Review and Meta-Analysis.

Understanding how older adults perceive and interpret emotional cues in speech prosody contributes to our knowledge of cognitive aging. This study provides a systematic review with meta-analysis to investigate the extent of the decline in affective prosody recognition (APR) among older adults in terms of overall and emotion-specific performance and explore potential moderators that may cause between-studies heterogeneity. The literature search encompassed five electronic databases, with a specific emphasis on studies comparing the APR performance of older adults with that of younger adults. This comparison was focused on basic emotions. Meta-regression analyses were executed to pinpoint potential moderators related to demographic and methodological characteristics. A total of 19 studies were included in the meta-analysis, involving 560 older adults with a mean age of 69.15 years and 751 younger adults with a mean age of 23.02 years. The findings indicated a substantial negative effect size (g = -1.21). Furthermore, the magnitude of aggregated effect sizes showed a distinct valence-related recognition pattern with positive prosody exhibiting smaller effect sizes. Language background and years of education were found to moderate the overall and emotion-specific (i.e., disgust and surprise) performance effect estimate, and age and gender significantly influenced the effect estimate of happiness. The results confirmed a significant decline in APR ability among older adults compared to younger adults, but this decline was unbalanced across basic emotions. Language background and educational level emerged as significant factors influencing older adults' APR ability. Moreover, participants with a higher mean age exhibited notably poorer performance in recognizing happy prosody. These findings underscore the need to further investigate the neurobiological mechanisms for APR decline associated with aging. https://doi.org/10.23641/asha.26407888.

CIRG-SL: Commonsense Inductive Relation Graph framework with Soft Labels for Empathetic Response Generation

Empathy, the ability to understand and respond to others’ emotions, is critical in dialogue systems, particularly for applications such as psychological counselling and casual conversation. However, existing approaches often struggle to accurately capture emotional transitions in user utterances and dialogue contexts across multiple levels of granularity. Additionally, the orthogonality of one-hot emotion representations overlooks the subtle relationships between emotion categories. To overcome these limitations, we propose a novel framework: the commonsense inductive relation graph with fine emotional soft labels for empathetic response generation. This framework refines the representation of user utterances by incorporating commonsense knowledge to derive external emotional cues, which are integrated into an adaptive inductive relation graph. This graph explicitly captures and models dynamic emotional trajectories throughout the dialogue. Additionally, we propose a fine-grained soft label construction method that more accurately reflects the nuanced distribution of emotions, replacing the rigid one-hot encoding with a continuous representation that better captures inter-emotion relationships. Our approach achieves strong performance on the benchmark EmpatheticDialogues dataset, with a perplexity score of 27.85, Dist-1 of 1.81, Dist-2 of 6.58, and a dialogue emotion accuracy of 55.65%. These results highlight the effectiveness of our framework in capturing subtle emotions and generating contextually rich, empathetic responses.

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

The impact of emotional expression by artificial intelligence recommendation chatbots on perceived humanness and social interactivity

Artificial intelligence-powered chatbots capable of expressing emotions have gained significant popularity in the realm of customer service. Although previous studies have explored the impact of emotional expression in chatbots, there is a lack of understanding regarding the precise effects of different emotional cues. In this study, we drew upon social presence theory to investigate how different emotional cues conveyed by recommendation chatbots affect perceived humanness, social interactivity, and social presence. We conducted a series of scenario-based online experiments to shed light on these dynamics. We found that all three emotional cues (text, emoticons, and images) employed by chatbots can increase perceived humanness and social interactivity. Social presence appears to be an underlying mechanism for these positive relationships. We also observed two-way interactions for any pair of emotional cues and a three-way interaction for all three emotional cues. Ultimately, to elicit the most favorable customer perception, we propose that a mode of emotional expression using either text or emoticons alone is most appropriate. These findings deepen our understanding of the impact of emotional expressions in chatbots and offer novel insights into how to deploy chatbots in customer service.

Perceived prolonged stress leads to difficulties in recognizing sadness from voice cues in men but not women.

It has long been known that stress has detrimental effects on cognition (e.g., Alderson & Novack, 2002; Lupien & Lepage, 2001), most notably documented for memory functions (e.g., Schwabe & Wolf, 2013). Interestingly, less is known about the effects of stress on other cognitive functions including language processing. Here, we have examined the effects of self-reported prolonged stress on recognition of emotional language content with a particular emphasis on gender differences. We tested how well 399 participants with different perceived stress levels recognized emotional voice cues. Findings confirm previous results from the emotional prosody literature by demonstrating that women generally outperform men in the vocal emotion recognition task. Crucially, results also revealed that medium levels of perceived stress impair the ability to detect sadness from voice cues in men but not women. These findings were not modulated by task demands (e.g., speeded response) or better acoustic discrimination abilities in women. Results are in line with the idea that perceived stress has a different impact on men versus women and that women have a higher level of experience in voice sadness recognition, potentially due to their predominant role as primary caretakers. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

What Is Faster than Where in Vocal Emotional Perception.

Voices carry a vast amount of information about speakers (e.g., emotional state; spatial location). Neuroimaging studies postulate that spatial ("where") and emotional ("what") cues are processed by partially independent processing streams. Although behavioral evidence reveals interactions between emotion and space, the temporal dynamics of these processes in the brain and its modulation by attention remain unknown. We investigated whether and how spatial and emotional features interact during voice processing as a function of attention focus. Spatialized nonverbal vocalizations differing in valence (neutral, amusement, anger) were presented at different locations around the head, whereas listeners discriminated either the spatial location or emotional quality of the voice. Neural activity was measured with ERPs of the EEG. Affective ratings were collected at the end of the EEG session. Emotional vocalizations elicited decreased N1 but increased P2 and late positive potential amplitudes. Interactions of space and emotion occurred at the salience detection stage: neutral vocalizations presented at right (vs. left) locations elicited increased P2 amplitudes, but no such differences were observed for emotional vocalizations. When task instructions involved emotion categorization, the P2 was increased for vocalizations presented at front (vs. back) locations. Behaviorally, only valence and arousal ratings showed emotion-space interactions. These findings suggest that emotional representations are activated earlier than spatial representations in voice processing. The perceptual prioritization of emotional cues occurred irrespective of task instructions but was not paralleled by an augmented stimulus representation in space. These findings support the differential responding to emotional information by auditory processing pathways.

Modulatory effects of goal relevance on emotional attention reveal that fear has a distinct value

ABSTRACT Threat-related stimuli can capture attention. However, it remains debated whether this capture is automatic or not. To address this question, we compared attentional biases to emotional faces using a dot-probe task (DPT) where emotion was never goal-relevant (Experiment 1) or made directly task-relevant by means of induction trials (Experiments 2–3). Moreover, the contingency between the DPT and induction trials was either partial (Experiment 2) or full (Experiment 3). Eye-tracking was used to ascertain that the emotional cue and the subsequent target were processed with peripheral vision. Experiments 1 and 2 both showed that negative faces captured attention, with faster target processing when it appeared on the same side as the preceding fearful face (i.e. fear-valid trials) compared to the opposite side where the neutral face was shown (i.e. fear-invalid trials), but also when it appeared on the side of the preceding neutral face (i.e. happy-invalid trials) compared to the happy face (i.e. happy-valid trials). Importantly, this preferential spatial orienting to negative emotion was not observed in Experiment 3, where the goal relevance of emotion was high. However, in that experiment, fearful faces produced a specific attentional bias during the DPT, which was mostly driven by the induction trials themselves.

The association between social rewards and anxiety: Links from neurophysiological analysis in virtual reality and social interaction game

Individuals’ affective experience can be intricate, influenced by various factors including monetary rewards and social factors during social interaction. However, within this array of factors, divergent evidence has been considered as potential contributors to social anxiety. To gain a better understanding of the specific factors associated with anxiety during social interaction, we combined a social interaction task with neurophysiological recordings obtained through an anxiety-elicitation task conducted in a Virtual Reality (VR) environment. Employing inter-subject representational similarity analysis (ISRSA), we explored the potential linkage between individuals’ anxiety neural patterns and their affective experiences during social interaction. Our findings suggest that, after controlling for other factors, the influence of the partner's emotional cues on individuals’ affective experiences is specifically linked to their neural pattern of anxiety. This indicates that the emergence of anxiety during social interaction may be particularly associated with the emotional cues provided by the social partner, rather than individuals’ own reward or prediction errors during social interaction. These results provide further support for the cognitive theory of social anxiety and extend the application of VR in future cognitive and affective studies.

AVaTER: Fusing Audio, Visual, and Textual Modalities Using Cross-Modal Attention for Emotion Recognition.

Multimodal emotion classification (MEC) involves analyzing and identifying human emotions by integrating data from multiple sources, such as audio, video, and text. This approach leverages the complementary strengths of each modality to enhance the accuracy and robustness of emotion recognition systems. However, one significant challenge is effectively integrating these diverse data sources, each with unique characteristics and levels of noise. Additionally, the scarcity of large, annotated multimodal datasets in Bangla limits the training and evaluation of models. In this work, we unveiled a pioneering multimodal Bangla dataset, MAViT-Bangla (Multimodal Audio Video Text Bangla dataset). This dataset, comprising 1002 samples across audio, video, and text modalities, is a unique resource for emotion recognition studies in the Bangla language. It features emotional categories such as anger, fear, joy, and sadness, providing a comprehensive platform for research. Additionally, we developed a framework for audio, video and textual emotion recognition (i.e., AVaTER) that employs a cross-modal attention mechanism among unimodal features. This mechanism fosters the interaction and fusion of features from different modalities, enhancing the model's ability to capture nuanced emotional cues. The effectiveness of this approach was demonstrated by achieving an F1-score of 0.64, a significant improvement over unimodal methods.

Physician’s prosodic behavior and the regulation of patient anxiety: A quantitative study within the communication accommodation theory framework

ObjectivesPatients commonly experience anxiety when visiting a medical provider. While regulating its level, the physician may adjust their own prosodic behavior to express different levels of affective empathy and shape the emotional outcome. The primary aim was to examine the impact of convergence and divergence with the patient’s prosodic emotional cues on the patient-perceived physician’s affective empathy and the emotion regulation process. MethodsAnalog patients (n = 132) made assessments after the exposure to an audio vignette of a physician – patient conversation in the prosodic convergence or divergence condition. ResultsConvergence with the prosodic cues of anxiety was associated with the physician experiencing the same as or a similar emotional state to the patient. The former type of emotional reaction was related to lower odds of anxiety alleviation. Prosodic convergence had a negative effect on anxiety alleviation through the mediation of experiencing the same emotional state as the patient. ConclusionsDivergence from the prosodic cues of anxiety and, therefore, a low patient-perceived level of this emotion may facilitate the anxiety regulation process. Practice implicationsA careful use of convergence and divergence with prosodic emotional cues and adequate control of prosody in general are strongly recommended for practitioners.

Emotional Overshadowing: Pleasant and Unpleasant Cues Overshadow Neutral Cues in Human Associative Learning.

The online version contains supplementary material available at 10.1007/s42761-024-00270-0.

Elevating sentiment analysis with VGG-16's facial expression insights

<span lang="EN-US">In today's data-driven world, the ability to analyze emotional responses is essential. The pressing necessity that drives this study is to revolutionize the field of sentiment analysis by extracting the hidden information from people's facial expressions. It examines people's preferences, worries, and pleasure, revealing their views on many topics. Beyond text-based sentiment analysis, this research adds facial expression-based sentiment analysis into existing systems for tailored recommendations and mental health monitoring. The system emphasizes visual stimuli's emotional influence to improve decision-making, content adaptability, and user experiences. The implementation involves transfer learning with the pre-trained VGG-16 model, which enhances ability to discern intricate emotional cues from facial expressions. Convolutional Neural Network (CNN) and contextual analysis allow the model to understand users' emotions and provide insights into their thoughts, feelings, and behaviours. To improve emotion recognition reliability and reactivity, this study examines Random Forest, Support Vector Machine (SVM), and CNN methodologies. The VGG-16 CNN model outperforms over SVM and Random Forest classifiers with accuracy of 95%. This study highlights facial expression-based sentiment analysis.</span>