Facial Emotion Research Articles

Emotional scenes as context in emotional expression recognition: The role of emotion or valence match.

Emotion recognition is influenced by contextual information such as social category cues or background scenes. However, past studies yielded mixed findings regarding whether broad valence or specific emotion matches drive context effects and how multiple sources of contextual information may influence emotion recognition. To address these questions, participants were asked to categorize expressions on male and female faces posing happiness and anger and happiness and fear on pleasant and fearful backgrounds (Experiment 1, conducted in 2019), fearful and disgusted expressions on fear and disgust eliciting backgrounds (Experiment 2, conducted in 2022), and fearful and sad expressions on fear and sadness eliciting backgrounds (Experiment 3, conducted in 2022). In Experiment 1 (where stimuli varied in valence), a broad valence match effect was observed. Faster recognition of happiness than fear and anger was more pronounced in pleasant compared to fearful scenes. In Experiments 2 and 3 (where stimuli were negative in valence), specific emotion match effects were observed. Faster recognition occurred when expression and background were emotionally congruent. In Experiments 1 and 3, poser sex independently moderated emotional expression recognition speed. These results suggest that the effect of emotional scenes on facial emotion recognition is mediated by a match in valence when broad valence is task-relevant. Specific emotion matches drive context effects when participants categorize expressions of a single valence. Looking at the influence of background contexts and poser sex together suggests that these two sources of contextual information have an independent rather than an interactive influence on emotional expression recognition speed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Introducing a novel dataset for facial emotion recognition and demonstrating significant enhancements in deep learning performance through pre-processing techniques

Facial expression recognition (FER) plays a pivotal role in various applications, ranging from human-computer interaction to psychoanalysis. To improve the accuracy of facial emotion recognition (FER) models, this study focuses on enhancing and augmenting FER datasets. It comprehensively analyzes the Facial Emotion Recognition dataset (FER13) to identify defects and correct misclassifications. The FER13 dataset represents a crucial resource for researchers developing Deep Learning (DL) models aimed at recognizing emotions based on facial features. Subsequently, this article develops a new facial dataset by expanding upon the original FER13 dataset. Similar to the FER + dataset, the expanded dataset incorporates a wider range of emotions while maintaining data accuracy. To further improve the dataset, it will be integrated with the extended Cohn-Kanade (CK+) dataset.This paper investigates the application of modern DL models to enhance emotion recognition in human faces. By training a new dataset, the study demonstrates significant performance gains compared with its counterparts. Furthermore, the article examines recent advances in FER technology and identifies critical requirements for DL models to overcome the inherent challenges of this task effectively. The study explores several DL architectures for emotion recognition in facial image datasets, with a particular focus on convolutional neural networks (CNNs). Our findings indicate that complex architecture, such as EfficientNetB7, outperforms other DL architectures, achieving a test accuracy of 78.9 %. Notably, the model surpassed the EfficientNet-XGBoost model, especially when used with the new dataset. Our approach leverages EfficientNetB7 as a backbone to build a model capable of efficiently recognizing emotions from facial images. Our proposed model, EfficientNetB7-CNN, achieved a peak accuracy of 81 % on the test set despite facing challenges such as GPU memory limitations. This demonstrates the model's robustness in handling complex facial expressions. Furthermore, to enhance feature extraction and attention mechanisms, we propose a new hybrid model, CBAM-4CNN, which integrates the convolutional block attention module (CBAM) with a custom 4-layer CNN architecture. The results showed that the CBAM-4CNN model outperformed existing models, achieving higher accuracy, precision, and recall metrics across multiple emotion classes. The results highlight the critical role of comprehensive and diverse data in enhancing model performance for facial emotion recognition.

Efficacy of a Culturally Adapted Emotion Recognition Training Program in Improving Facial Emotion Recognition in Persons with Schizophrenia

Background: Facial emotion recognition (FER) deficits are being considered as core features of social cognitive deficits in schizophrenia, associated with socio-occupational dysfunction. Social cognition intervention programs have been shown to improve these deficits; however, there are no such intervention strategies in India. In this study, we aim to examine the efficacy of Training of Affect Recognition – Indian Version (TAR-IV) to enhance the FER abilities of people with schizophrenia. Methods: In an open-label experimental design, 36 participants with schizophrenia underwent 12 sessions of TAR-IV as an add-on to treatment as usual (TAU), while 29 participants with schizophrenia continued TAU (pharmacological treatment with or without occupational therapy and vocational rehabilitation services). Clinical and functional assessments were done using the positive and negative syndrome scale (PANSS) and socio-occupational functioning scale. Emotion recognition abilities were assessed on the tool for recognition of emotions in neuropsychiatric disorders at baseline, post-intervention, and follow-up (two to three months post-intervention). Results: The intervention group showed significant improvements in FER ( P = .001) and socio-occupational functioning ( P = .008) after receiving the TAR-IV, which remained significant at two months follow-up. A lower age of onset and poorer neurocognitive function at baseline predicted greater changes in emotion recognition ability following the intervention. Conclusions: This study demonstrated the efficacy of TAR-IV, the Indian adaptation of social cognition intervention, in improving emotion recognition abilities and socio-occupational functioning in patients with schizophrenia. These findings need to be replicated in randomized controlled trials.

Neural Representations of Emotions in Visual, Auditory, and Modality-Independent Regions Reflect Idiosyncratic Conceptual Knowledge.

Growing evidence suggests that conceptual knowledge influences emotion perception, yet the neural mechanisms underlying this effect are not fully understood. Recent studies have shown that brain representations of facial emotion categories in visual-perceptual areas are predicted by conceptual knowledge, but it remains to be seen if auditory regions are similarly affected. Moreover, it is not fully clear whether these conceptual influences operate at a modality-independent level. To address these questions, we conducted a functional magnetic resonance imaging study presenting participants with both facial and vocal emotional stimuli. This dual-modality approach allowed us to investigate effects on both modality-specific and modality-independent brain regions. Using univariate and representational similarity analyses, we found that brain representations in both visual (middle and lateral occipital cortices) and auditory (superior temporal gyrus) regions were predicted by conceptual understanding of emotions for faces and voices, respectively. Additionally, we discovered that conceptual knowledge also influenced supra-modal representations in the superior temporal sulcus. Dynamic causal modeling revealed a brain network showing both bottom-up and top-down flows, suggesting a complex interplay of modality-specific and modality-independent regions in emotional processing. These findings collectively indicate that the neural representations of emotions in both sensory-perceptual and modality-independent regions are likely shaped by each individual's conceptual knowledge.

The impact of stimulus format and presentation order on social working memory updating.

Emotional faces and words have been extensively employed to examine cognitive emotional processing including social working memory, which plays a pivotal role in social interactions. However, it remains unclear which exact role these two stimulus formats play in updating specific emotional content, such as positive or negative information. Therefore, the current study examined the differences in working memory updating (WMU) of negative, neutral, and positive faces (Experiment 1) and words (Experiment 2), using a classic two-back paradigm with an event-related potential technique. In both experiments, emotional stimuli were presented in the same or different-valence order to further determine whether presentation order can also influence the WMU of specific emotional content. Our results showed that both stimulus format and presentation order play a role: (a) while faces showed an affective bias [larger P2 and late positive potential (LPP) for negative and positive faces than for neutral faces], words showed a negativity bias (larger LPP for negative words than both neutral and positive words); (b) While faces showed better performance with same-valence order, words showed better performance with different-valence order. Taken together, our findings indicate that, even if faces and words can contain the same emotional information, they impact social WMU differently.

Facial Expression Recognition Based on the FER2013 Dataset

Facial emotions are a way to express one's thoughts and also an effective way to understand the emotions of others. Nowadays, with the rapid development of technology, computers can also recognize facial expressions through convolutional neural networks, deep learning, and other methods, and classify the results. Throughout the entire experiment, we chose FER2013 data as the training set for our model, which ultimately achieved an accuracy of around 62%. We also compared it with the SFEW dataset. The emergence of facial expression recognition will increase in the future, and its application in teaching supervision is what we are exploring here. Its main function can be used for invigilation, attendance, checking class status, and so on.

Validity Study of an Emotional Face-Database in Iranian Community

Objective: To investigate the accuracy of facial emotion recognition in the Iranian community, a face database validated in this community is required. To do this, we conducted a validation study on the Radboud face database. The primary objective of this study was to evaluate the accuracy of recognizing emotions through faces in an Iranian sample and then to choose the pictures with high agreement in terms of detecting emotions. Method: This is a cross sectional study recruiting a total number of 142 males and females aged between 20 and 50 years old (Mean ± SD of age 31.7 ± 7.07). The participants were instructed to detect the type of emotion of each face as well as its valence and arousal. The percentage of participants’ agreement on evaluating each picture was assessed. To evaluate the effect of different variables on participants’ accuracy, one way and repeated measure ANOVA analyses were also used. Results: Emotional faces were recognized by around 84% of the participants. The highest accuracy belongs to happy (Mean ± SD of 98 ± 6.1%) and the lowest one to neutral (75 ± 18.06%) faces. The accuracy for detecting other emotions were as follows: sad (91 ± 8.7%), surprised (87 ± 10.64%), angry (77 ± 15.6%), and fearful (76 ± 15.26%). Additionally, we found no differences between male and female participants in terms of recognizing emotions. Then we selected the pictures with high agreement (above 85 percent) in labeling emotions among the participants. Conclusion: The current study provided a valid emotional face database based on Iranian participants’ responses in terms of recognizing basic emotions. The selected pictures can be used in designing tasks to evaluate emotion recognition ability in clinical and nonclinical populations. It can also be used in designing applications to improve detecting emotion in clinical samples such as individuals with autism spectrum disorder.

Cross-channel adaptation reveals shared emotion representation from face and biological motion.

Emotions in interpersonal interactions can be communicated simultaneously via various social signals such as face and biological motion (BM). Here, we demonstrate that even though BM and face are very different in visual properties, emotions conveyed by these two types of social signals involve dedicated and common processing mechanisms (N = 168, college students, 2020-2024). By utilizing the visual adaptation paradigm, we found that prolonged exposure to the happy BM biased the emotion perception of the subsequently presented morphed BM toward sad, and vice versus. The observed aftereffect disappeared when the BM adaptors were shown inverted, indicating that it arose from emotional information processing rather than being a result of adaptation to constitutive low-level features. Besides, such an aftereffect was also found for facial expressions and similarly vanished when the face adaptors were inverted. Critically, preexposure to emotional faces also exerted an adaptation aftereffect on the emotion perception of BMs. Furthermore, this cross-channel effect could not only happen from faces to BMs but also from BMs to faces, suggesting that emotion perception from face and BM are potentially driven by common underlying neural substrates. Overall, these findings highlighted a close coupling of BM and face emotion perception and suggested the existence of a dedicated emotional representation that can be shared across these two different types of social signals. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The Application of Emotion Valence Ratios in Facial Emotion Recognition for Detecting Depression Among Older Adults in Institutional Settings.

Developing evidence-based parameters to enhance the reliability of face emotion recognition (FER) systems in detecting depression among the elderly is essential. This study aims to elucidate the relationship between the ratio of each emotion valence captured by the FER system and heart rate variability (HRV) while participants watch a video in relation to their depression scores. YOLO, an open-source data analysis toolkit, was used to extract three facial emotion valence features (neutral, positive, and negative) and determine the ratio of each emotion valence over time during video viewing. Additionally, HRV was assessed, and the Geriatric Depression Scale was administered to understand the correlation between FER parameters and depression scores.

Detecting five-pattern personality traits using eye movement features for observing emotional faces.

The five-pattern personality traits rooted in the theory of traditional Chinese medicine (TCM) have promising prospects for clinical application. However, they are currently assessed using a self-report scale, which may have certain limitations. Eye tracking technology, with its non-intrusive, objective, and culturally neutral characteristics, has become a powerful tool for revealing individual cognitive and emotional processes. Therefore, applying this technology for personality assessment is a promising approach. In this study, participants observed five emotional faces (anger, happy, calm, sad, and fear) selected from the Chinese Facial Affective Picture System. Utilizing artificial intelligence algorithms, we evaluated the feasibility of automatically identifying different traits of the five-pattern personality traits from participants' eye movement patterns. Based on the analysis of five supervised learning algorithms, we draw the following conclusions: The Lasso feature selection method and Logistic Regression achieve the highest prediction accuracy for most of the traits (TYa, SYa, SYi, TYi). This study develops a framework for predicting five-pattern personality traits using eye movement behavior, offering a novel approach for personality assessment in TCM.

Comparative study of emotional facial expression recognition among Prader-Willi syndrome subtypes.

Prader-Willi syndrome (PWS) is a congenital disease caused by a rare and generally non-inherited genetic disorder. The inability to recognise facial expressions of emotion is an apparent social cognition deficit in people diagnosed with PWS. The main objective of the present study is to compare the ability to recognise emotional facial expression, in both non-contextualised and contextualised scenarios, among the main subtypes of PWS and a control group. The sample consisted of 46 children divided into three groups: deletion (n=10), maternal uniparental disomy (mUPD) (n=13) and control (n=23). The protocol included the Facially Expressed Emotion Labeling and the Deusto-e-Motion 1.0. The control group recognised facial emotions more accurately and quickly in both non-contextualised and contextualised scenarios than children with PWS, regardless of genetic subtype. Despite no differences being detected between PWS subtypes when non-contextualised scenarios were analysed, in contextualised situations, a longer reaction time was observed in children with the mUPD subtype. This is the first study to assess the ability to recognise emotional facial expressions in contextualised situations among PWS subtypes and a control group. The findings suggest that some of the social cognitive deficits evidenced in children with mUPD PWS may be similar to those in autism spectrum disorder.

Impaired facial emotion recognition in individuals with bipolar disorder

BackgroundIndividuals with bipolar disorder (BD) often struggle with emotional regulation and social interactions, partly due to difficulties in accurately recognizing facial emotions. MethodsFrom September 2021 to February 2023, 69 BD individuals-comprising 23 with bipolar manic/hypomanic episode (BME), 23 with bipolar depressive episode (BDE), 23 with bipolar euthymic (EUT)-and 23 healthy controls (HCs) were enrolled. Diagnosis adhered to DSM-IV criteria using M.I.N.I 5.0, alongside assessments via Hamilton Depression Scale 17 and Young Manic Rating Scale. Recognition tasks involving 84 facial expression images across six categories. The Wilcoxon rank-sum test compares two groups, while the Kruskal-Wallis test compares multiple groups with subsequent adjusted pairwise comparisons. ResultsThe overall correct recognition rate of facial expressions in the BD group (79 %) was significantly lower than that of the HC group (83 %) (P=0.004). Primary differences were noted in neutral (93 % vs. 100 %, P=0.012) and fear (79 % vs. 86 %, P=0.023) expressions. Within the BD group, correct recognition rates were 71 % for BME, 80 % for BDE, and 80 % for EUT, all lower than in the HC group. Significant differences in correct recognition rates of neutral, fear, and joy expressions were observed among the four groups (P<0.05), with the BME group exhibiting the lowest rate. Misidentification of facial expressions was more frequent in the BD group compared to the HC group, particularly among negative expressions. ConclusionPatients with BD demonstrate lower correct recognition and higher misidentification rates of facial expressions, with those experiencing manic episodes showing impaired recognition of neutral, joy, and fear expressions.

Individuals with high autism traits show top-down attention bias towards threatening stimuli

Individuals with autistic traits (AT) are widely distributed in the general population. Strengthening understanding of AT can provide a broader perspective for autism research as well as more accurate diagnosis and personalized treatment plans for clinical practice. Previous studies on attention bias among high-AT individuals have yielded inconsistent results, which may relate to different stages of attention. In this study, we selected two groups with high and low level AT from the general population, and then adopted the odd-one-out search task, combined with Event-related potential (ERP) technique, conducted both attention orientation and attention dissociation tasks, to explore attention bias and electroencephalogram (EEG) characteristics towards threatening emotional faces in these groups. The behavioral data showed no accelerated attention orientation to angry faces and neutral faces; however, there was attention dissociation difficulty for angry faces. Compared with low-AT individuals, the EEG results showed that high-AT individuals have acceleration of attention orientation and attention dissociation difficulty for threatening emotional faces. From the perspective of top-down concept-driven processing, these findings suggest that high-AT individuals have attention bias for cognitive processing of threatening stimuli, which is mainly due to acceleration of attention orientation and attention dissociation difficulties for threatening information.

Beyond the Whole: Reduced Empathy for Masked Emotional Faces Is Not Driven by Disrupted Configural Face Processing.

Sharing of emotional states is reduced for individuals wearing face coverings, but the mechanism behind this reduction remains unknown. Here, we investigated if face occlusion by masks reduces empathy by disrupting configural processing of emotional faces. Participants rated their empathy for happy and neutral faces which were presented in upright or inverted orientation and wore opaque, clear, or no face masks. Empathy ratings were reduced for masked faces (opaque or clear) as well as for inverted faces. Importantly, face inversion disrupted empathy more for faces wearing opaque masks relative to those wearing clear or no masks, which stands in contrast to the predictions generated by the classic configural processing models. We discuss these data within the context of classic and novel configural face perception models, and highlight that studying inverted occluded faces presents an informative case worthy of further investigation.

Modulatory effects of goal relevance on emotional attention reveal that fear has a distinct value

ABSTRACT Threat-related stimuli can capture attention. However, it remains debated whether this capture is automatic or not. To address this question, we compared attentional biases to emotional faces using a dot-probe task (DPT) where emotion was never goal-relevant (Experiment 1) or made directly task-relevant by means of induction trials (Experiments 2–3). Moreover, the contingency between the DPT and induction trials was either partial (Experiment 2) or full (Experiment 3). Eye-tracking was used to ascertain that the emotional cue and the subsequent target were processed with peripheral vision. Experiments 1 and 2 both showed that negative faces captured attention, with faster target processing when it appeared on the same side as the preceding fearful face (i.e. fear-valid trials) compared to the opposite side where the neutral face was shown (i.e. fear-invalid trials), but also when it appeared on the side of the preceding neutral face (i.e. happy-invalid trials) compared to the happy face (i.e. happy-valid trials). Importantly, this preferential spatial orienting to negative emotion was not observed in Experiment 3, where the goal relevance of emotion was high. However, in that experiment, fearful faces produced a specific attentional bias during the DPT, which was mostly driven by the induction trials themselves.

Facial emotion recognition in children with attention deficit hyperactivity disorder

Objective Attention Deficit Hyperactivity Disorder (ADHD) is defined as a persistent pattern of inattention and hyperactivity-impulsivity that interferes with functioning anofd development. Increased emotional reactivity and impaired emotion regulation are established findings in children with ADHD. Impairments in executive functions such as impulse control and working memory, in turn, have also been suggested to have a negative effect on emotion recognition. However, studies exploring suspected deficits in the ability to recognise facial emotions in ADHD have to date yielded controversial results. We sought to clarify the mechanism of possible emotion recognition dysfunction in children with ADHD. Methods Sixty-one children diagnosed with ADHD (aged 10.36 ± 1.89 years) and a control group (N = 78; aged 9.6 ± 1.8 years) were evaluated with questionnaires and computerized tests for cognitive and facial emotion recognition capacity. Results The ADHD group displayed more behavioural issues and performed worse in cognitive tests compared to the control group. Group status (i.e. ADHD vs. control group) did not predict facial emotion recognition when controlled for age, IQ and sex in linear regression models. Performance in Divided Attention predicted facial emotion recognition in linear regression in the ADHD group. Conclusions Individuals with ADHD showed facial emotion recognition capacity similar to a typically developing control group. Good performance in a cognitive test assessing divided attention predicted capacity for facial emotion recognition, but only in the ADHD group.

The neuronal signature of surprised facial expression processing under different attentional focuses: A time-domain and time-frequency study.

Previous studies have shown that high-arousal positive and negative facial expressions influence event-related potential (ERP) and time-frequency responses depending on attentional focuses. However, little is known about how relevant neural responses are influenced by surprised facial expressions, which are also high in arousal but ambiguous in valence. To address the issue, 38 participants were presented with surprised, happy, angry and neutral facial expressions. Attention was manipulated to focus on facial emotional attributes, facial non-emotional attributes, non-facial attributes, or was free to the participants. ERP results showed larger N170 responses to surprised compared to neutral facial expressions when attention focused on facial attributes and to surprised compared to angry and neutral facial expressions when attention focused on facial non-emotional attributes. Time-frequency analyses revealed reduced power of early occipital theta to surprised compared to happy and angry expressions when attention focused on facial emotions and to surprised compared to angry expressions when attention focused on non-facial stimuli. Parietal delta power was smaller for surprised facial expressions than for angry facial expressions when attention focused on facial emotions and for surprised facial expressions than for angry and neutral facial expressions when attention was directed to non-facial stimuli. These findings might suggest that neural responses to surprised facial expressions are modulated by attentional focus.

Road lighting for pedestrians: Comparing full-face and 3/4 views in a facial emotion recognition task

Road lighting should support the needs of pedestrians to make interpersonal evaluations after dark, for example, whether it feels safe to walk towards the person ahead or if avoiding action should be taken. In previous studies this has been investigated using a facial emotion recognition (FER) task, but using only full-face views of the person ahead. Other views are possible and this might affect the ability to make FER judgements and thus the impact of changes in light level. Reported here are the results of an experiment investigating FER with full-face and 3/4 views. The results show that while correct expression recognition is reduced with the 3/4 view, there is no interaction between face luminance and face view.

Facial Emotion Recognition for Feature Extraction and Ensemble Learning Using Hierarchical Cascade Regression Neural Networks and Random Forest

Facial expression detection technology is a flexible tool utilized in many different industries that allows for a better understanding of human behavior and emotional responses. Its applications span a broad spectrum of societal requirements, from enhancing marketing strategies to bolstering security protocols and assisting in the diagnosis of mental health issues. By using the power of artificial intelligence and computer vision, facial expression recognition (FER) helps businesses to adapt, grow and build deeper interactions between people and the technology and environments they live in. Its primary goal remains to facilitate better understanding, communication and decision-making in order to bring in a new wave of technical innovation that is driven by people’s needs. Facial emotion recognition, which makes use of computer vision and artificial intelligence, not only improves the capacity to understand and react to human emotions but also opens the door to more personalized and empathic interactions in an increasingly digital environment. In this paper, unique method of facial emotion recognition based on sophisticated feature extraction techniques combined with a hybrid ensemble learning technique. In particular, hierarchical cascade regression neural networks (HCRNN) for face landmark identification, VGG-19 for feature extraction and Random Forests are considered as classification techniques. In order to accomplish reliable and accurate emotion recognition from facial images, this method takes advantage of the complementary capabilities of both approaches. Initially, we utilized a deep convolutional neural network (CNN) called VGG-19 to extract high-level features from face images. Pre-trained on extensive image datasets, VGG-19 has proven to be highly effective at capturing complex visual representations. These characteristics function as an all-inclusive description of facial expressions, encompassing both general trends and minute details. HCRNN for facial landmark detection, which precisely locates important face features including the mouth, nose and eyes. Facial landmark predictions are iteratively improved by this hierarchical cascade architecture, which also handles occlusions and changes in illumination and position. Lastly, we use an ensemble learning technique called Random Forests to classify emotions. Multiple decision trees’ predictions are combined by random forests, which offer robustness against overfitting and efficiently handle high-dimensional feature fields. High accuracy and generalization capabilities are achieved in the facial emotion classification process by Random Forests, which combine the feature representations retrieved by VGG-19 with the localized facial landmarks recognized by HCRNN. We tested the suggested strategy on benchmark face emotion detection datasets and compared its results with the most advanced techniques available. Performance measures like accuracy of 98.89%, precision 96.52%, recall 96%, F1-score 99% and receiver operating characteristic (ROC) curve of 97.24% analysis were used to evaluate how well hybrid strategy worked and how much better it was at identifying emotions in facial images. The outcomes of experiments show how method has practical applications in human–computer interaction, healthcare and other fields.

Analyzing deep textual facial patterns for human pain sentiment recognition system in smart healthcare framework

BACKGROUND: Patient sentiment analysis aids in identifying issue areas, timely remediation, and improved patient care by the healthcare professional. The relationship between pain management and patient sentiment analysis is crucial to providing patients with high-quality medical care. Therefore, a self-reported pain level assessment is required for the smart healthcare framework to determine the best course of treatment. OBJECTIVE: An efficient method for a pain sentiment recognition system has been proposed based on the analysis of human facial emotion patterns of patients in the smart healthcare framework. METHODS: The proposed system has been implemented in four phases: (i) in the first phase, the facial regions of the observation patient have been detected using the computer vision-based face detection technique; (ii) in the second phase, the extracted facial regions are analyzed using deep learning based feature representation techniques to extract discriminant and crucial facial features to analyze the level of pain emotion of patient; (iii) the level of pain emotions belongs from macro to micro facial expressions, so, some advanced feature tunning and representation techniques are built along with deep learning based features such as to distinguish low to high pain emotions among the patients in the third phase of the implementation, (iv) finally, the performance of the proposed system is enhanced using the score fusion techniques applied on the obtained deep pain recognition models for the smart healthcare framework. RESULTS: The performance of the proposed system has been tested using two standard facial pain benchmark databases, the UNBC-McMaster shoulder pain expression archive dataset and the BioVid Heat Pain Dataset, and the results are compared with some existing state-of-the-art methods employed in this research area. CONCLUSIONS: From extensive experiments and comparative studies, it has been concluded that the proposed pain sentiment recognition system performs remarkably well compared to the other pain recognition systems for the smart healthcare framework.