Low Intensity Expressions Research Articles

A Deep Learning Approach for Human Facial Expression Recognition using Residual Network – 101

Emotion recognition is a dynamic process that focuses on a person's emotional state, which implies that the emotions associated with each individual's activities are unique. Human emotion analysis and recognition have been popular study areas among computer vision researchers. High dimensionality, execution time, and cost are the main difficulties in human emotion detection. To deal with these issues, the proposed model aims to design a human emotion recognition model using Residual Networks-101 (ResNet-101). A Convolutional Neural Network (CNN) design called ResNet-101 solves the vanishing gradient issue and makes it possible to build networks with thousands of convolutional layers that outperform networks with fewer layers. An image dataset was used for this emotion recognition. Then, this image dataset was subjected to preprocessing to resize the image and eliminate the noise contents present in the images. After preprocessing, the image was given to the classifier to recognize the emotions effectively. Here, ResNet-101 was used for the classification of six classes. The experimental results demonstrate that ResNet-101 models outperform the most recent techniques for emotion recognition. The proposed model was executed in MATLAB software and carried out several performance metrics. The proposed architecture attained better performance in terms of accuracy 92% and error with 0.08 and other performances like 92% of precision, 85% of specificity and 98% of sensitivity so on, and this shows the effectiveness of the proposed model to existing approaches such as LeNet, AlexNet and VGG. In comparison to current techniques, the suggested model provides improved recognition accuracy for low intensity or mild emotional expressions.

Face masks affect perception of happy faces in deaf people

The SARS-CoV-2 pandemic has led significant social repercussions and forced people to wear face masks. Recent research has demonstrated that the human ability to infer emotions from facial configurations is significantly reduced when face masks are worn. Since the mouth region is specifically crucial for deaf people who speak sign language, the current study assessed the impact of face masks on inferring emotional facial expressions in a population of adult deaf signers. A group of 34 congenitally deaf individuals and 34 normal-hearing individuals were asked to identify happiness, sadness, fear, anger, and neutral expression on static human pictures with and without facial masks presented through smartphones. For each emotion, the percentage of correct responses with and without face masks was calculated and compared between groups. Results indicated that face masks, such as those worn due to the SARS-CoV-2 pandemic, limit the ability of people to infer emotions from facial expressions. The negative impact of face masks is significantly pronounced when deaf people have to recognize low-intensity expressions of happiness. These findings are of essential importance because difficulties in recognizing emotions from facial expressions due to mask wearing may contribute to the communication challenges experienced by the deaf community during the SARS-CoV-2 pandemic, generating feelings of frustration and exclusion.

Sensitivity to Emotion Intensity and Recognition of Emotion Expression in Neurotypical Children.

This study assessed two components of face emotion processing: emotion recognition and sensitivity to intensity of emotion expressions and their relation in children age 4 to 12 (N = 216). Results indicated a slower development in the accurate decoding of low intensity expressions compared to high intensity. Between age 4 and 12, children discriminated high intensity expressions better than low ones. The intensity of expression had a stronger impact on overall face expression recognition. High intensity happiness was better recognized than low intensity up to age 11, while children 4 to 12 had difficulties discriminating between high and low intensity sadness. Our results suggest that sensitivity to low intensity expressions acts as a complementary mediator between age and emotion expression recognition, while this was not the case for the recognition of high intensity expressions. These results could help in the development of specific interventions for populations presenting socio-cognitive and emotion difficulties.

Recognition of vocal socioemotional expressions at varying levels of emotional intensity.

Nonverbal expressions of emotion can vary in intensity, from ambiguous to prototypical exemplars: for instance, facial displays of happiness may range from a faint smile to a full-blown grin. Previous work suggests that the accuracy with which facial expressions are recognized as the intended emotion increases with emotional intensity, although this pattern depends on the displayed emotion. Less is known about the association between emotional intensity and the recognition of vocal emotional expressions (affective prosody), which also convey information about others' socioemotional intent but are perceived and interpreted differently than facial expressions. The current study examined listeners' ability to recognize emotional intent in morphed vocal prosody recordings that varied in emotional intensity from neutral to prototypical exemplars of basic emotions (anger, disgust, fear, happiness, sadness) and social expressions (friendliness, meanness). Results suggest that listeners' accuracy in identifying the intended emotional intent in each recording increased nonlinearly with emotional intensity. This pattern varied by emotion type: for instance, accuracy for anger rose steeply with increasing emotional intensity before plateauing, whereas accuracy for happiness remained unchanged across low-intensity exemplars but increased thereafter. These findings highlight emotion-specific ways in which dynamic changes in emotional intensity inform perceptions of socioemotional intent in emotional prosody. Moreover, these results also point to potential challenges in emotional communication in social interactions that rely primarily on the voice, with many low-intensity expressions having a higher probability of being misinterpreted. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A Compact Deep Learning Model for Robust Facial Expression Recognition

In this paper we are proposing a compact CNN model for facial expression recognition. Expression recognition on the low quality images are much more challenging and interesting due to the presence of low-intensity expressions. These low intensity expressions are difficult to distinguish with insufficient image resolution. Data collection for FER is expensive and time-consuming. Researches indicates the fact that downloaded images from the Internet is very useful to model and train expression recognition problem. We use extra datasets to improve the training of facial expression recognition, each representing specific data source. Moreover, to prevent subjective annotation, each dataset is labeled with different approaches to ensure annotation qualities. Recognizing the precise and exact expression from a variety of expressions of different people is a huge problem. To solve this problem, we proposed an Emotion Detection Model to extract emotions from the given input image. This work mainly focuses on the psychological approach of color circle-emotion relation[1] to find the accurate emotion from the input image. Initially the whole image is preprocessed and pixel by pixel data is studied. And the combinations of the circles based on combined data will result in a new color. This resulted color will be directly correlated to a particular emotion. Based on the psychological aspects the output will be of reasonable accuracy. The major application of our work is to predict a person’s emotion based on his face images or video frames This can even be applied for evaluating the public opinion relating to a particular movie, form the video reaction posts on social Medias. One of the diverse applications of our system is to understand the students learning from their emotions. Human beings shows their emotional states and intentions through facial expressions.. Facial expressions are powerful and natural methods that emphasize the emotional status of humans .The approach used in this work successfully exploits temporal information and it improves the accuracies on the public benchmarking databases. The basic facial expressions are happiness, fear, anger, disgust sadness, and surprise[2]. Contempt was subsequently added as one of the basic emotions. Having sufficient well labeled training data with variations of the populations and environments is important for the design of a deep expression recognition system .Behaviors, poses, facial expressions, actions and speech are considered as channels, which convey human emotions. Lot of research works are going on in this field to explore the correlation between the above mentioned channels and emotions. This paper highlights on the development of a system which automatically recognizes the

A Compact Deep Learning Model for Robust Facial Expression Recognition.

In this paper we are proposing a compact CNN model for facial expression recognition. Expression recognition on the low quality images are much more challenging and interesting due to the presence of low-intensity expressions. These low intensity expressions are difficult to distinguish with insufficient image resolution. Data collection for FER is expensive and time-consuming. Researches indicates the fact that downloaded images from the Internet is very useful to model and train expression recognition problem. We use extra datasets to improve the training of facial expression recognition, each representing specific data source. Moreover, to prevent subjective annotation, each dataset is labeled with different approaches to ensure annotation qualities. Recognizing the precise and exact expression from a variety of expressions of different people is a huge problem. To solve this problem, we proposed an Emotion Detection Model to extract emotions from the given input image. This work mainly focuses on the psychological approach of color circle-emotion relation[1] to find the accurate emotion from the input image. Initially the whole image is preprocessed and pixel by pixel data is studied. And the combinations of the circles based on combined data will result in a new color. This resulted color will be directly correlated to a particular emotion. Based on the psychological aspects the output will be of reasonable accuracy. The major application of our work is to predict a person’s emotion based on his face images or video frames This can even be applied for evaluating the public opinion relating to a particular movie, form the video reaction posts on social Medias. One of the diverse applications of our system is to understand the students learning from their emotions. Human beings shows their emotional states and intentions through facial expressions.. Facial expressions are powerful and natural methods that emphasize the emotional status of humans .The approach used in this work successfully exploits temporal information and it improves the accuracies on the public benchmarking databases. The basic facial expressions are happiness, fear, anger, disgust sadness, and surprise[2]. Contempt was subsequently added as one of the basic emotions. Having sufficient well labeled training data with variations of the populations and environments is important for the design of a deep expression recognition system .Behaviors, poses, facial expressions, actions and speech are considered as channels, which convey human emotions. Lot of research works are going on in this field to explore the correlation between the above mentioned channels and emotions. This paper highlights on the development of a system which automatically recognizes the emotion represented on a face. A neural network approach combined with digital image processing is used in classifying the universal basic emotions such as Happiness, Sadness, Anger, Disgust, Surprise and Fear. Colored frontal face images are given as input to the proposed system. The input face image is preprocessed and feature point extraction method is applied to extract a set of selected feature points. Finally, the set of values obtained after processing those extracted feature points are taken as input to the neural network to recognize the emotion contained .The three major steps in any automatic deep face emotion recognition model are pre-processing, deep feature learning and deep feature classification.

Caricaturing can improve facial expression recognition in low-resolution images and age-related macular degeneration.

Previous studies of age-related macular degeneration (AMD) report impaired facial expression recognition even with enlarged face images. Here, we test potential benefits of caricaturing (exaggerating how the expression's shape differs from neutral) as an image enhancement procedure targeted at mid- to high-level cortical vision. Experiment 1 provides proof-of-concept using normal vision observers shown blurred images as a partial simulation of AMD. Caricaturing significantly improved expression recognition (happy, sad, anger, disgust, fear, surprise) by ∼4%-5% across young adults and older adults (mean age 73 years); two different severities of blur; high, medium, and low intensity of the original expression; and all intermediate accuracy levels (impaired but still above chance). Experiment 2 tested AMD patients, running 19 eyes monocularly (from 12 patients, 67-94 years) covering a wide range of vision loss (acuities 6/7.5 to poorer than 6/360). With faces pre-enlarged, recognition approached ceiling and was only slightly worse than matched controls for high- and medium-intensity expressions. For low-intensity expressions, recognition of veridical expressions remained impaired and was significantly improved with caricaturing across all levels of vision loss by 5.8%. Overall, caricaturing benefits emerged when improvement was most needed, that is, when initial recognition of uncaricatured expressions was impaired.

Age-related differences in emotion matching are limited to low intensity expressions

ABSTRACTMulti-label tasks confound age differences in perceptual and cognitive processes. We examined age differences in emotion perception with a technique that did not require verbal labels. Participants matched the emotion expressed by a target to two comparison stimuli, one neutral and one emotional. Angry, disgusted, fearful, happy, and sad facial expressions of varying intensity were used. Although older adults took longer to respond than younger adults, younger adults only outmatched older adults for the lowest intensity disgust and fear expressions. Some participants also completed an identity matching task in which target stimuli were matched on personal identity instead of emotion. Although irrelevant to the judgment, expressed emotion still created interference. All participants were less accurate when the apparent difference in expressive intensity of the matched stimuli was large, suggesting that salient emotion cues increased difficulty of identity matching. Age differences in emotion perception were limited to very low intensity expressions.

Impaired Recognition of Basic Emotions from Facial Expressions in Young People with Autism Spectrum Disorder: Assessing the Importance of Expression Intensity

It has been proposed that impairments in emotion recognition in ASD are greater for more subtle expressions of emotion. We measured recognition of 6 basic facial expressions at 8 intensity levels in young people (6–16 years) with ASD (N = 63) and controls (N = 64) via an Internet platform. Participants with ASD were less accurate than controls at labelling expressions across intensity levels, although differences at very low levels were not detected due to floor effects. Recognition accuracy did not correlate with parent-reported social functioning in either group. These findings provide further evidence for an impairment in recognition of basic emotion in ASD and do not support the idea that this impairment is limited solely to low intensity expressions.

Validation of the Amsterdam Dynamic Facial Expression Set--Bath Intensity Variations (ADFES-BIV): A Set of Videos Expressing Low, Intermediate, and High Intensity Emotions.

Most of the existing sets of facial expressions of emotion contain static photographs. While increasing demand for stimuli with enhanced ecological validity in facial emotion recognition research has led to the development of video stimuli, these typically involve full-blown (apex) expressions. However, variations of intensity in emotional facial expressions occur in real life social interactions, with low intensity expressions of emotions frequently occurring. The current study therefore developed and validated a set of video stimuli portraying three levels of intensity of emotional expressions, from low to high intensity. The videos were adapted from the Amsterdam Dynamic Facial Expression Set (ADFES) and termed the Bath Intensity Variations (ADFES-BIV). A healthy sample of 92 people recruited from the University of Bath community (41 male, 51 female) completed a facial emotion recognition task including expressions of 6 basic emotions (anger, happiness, disgust, fear, surprise, sadness) and 3 complex emotions (contempt, embarrassment, pride) that were expressed at three different intensities of expression and neutral. Accuracy scores (raw and unbiased (Hu) hit rates) were calculated, as well as response times. Accuracy rates above chance level of responding were found for all emotion categories, producing an overall raw hit rate of 69% for the ADFES-BIV. The three intensity levels were validated as distinct categories, with higher accuracies and faster responses to high intensity expressions than intermediate intensity expressions, which had higher accuracies and faster responses than low intensity expressions. To further validate the intensities, a second study with standardised display times was conducted replicating this pattern. The ADFES-BIV has greater ecological validity than many other emotion stimulus sets and allows for versatile applications in emotion research. It can be retrieved free of charge for research purposes from the corresponding author.

Feedback training induces a bias for detecting happiness or fear in facial expressions that generalises to a novel task

Many psychological disorders are characterised by insensitivities or biases in the processing of subtle facial expressions of emotion. Training using expression morph sequences which vary the intensity of expressions may be able to address such deficits. In the current study participants were shown expressions from either happy or fearful intensity morph sequences, and trained to detect the target emotion (e.g., happy in the happy sequence) as being present in low intensity expressions. Training transfer was tested using a six alternative forced choice emotion labelling task with varying intensity expressions, which participants completed before and after training. Training increased false alarms for the target emotion in the transfer task. Hit rate for the target emotion did not increase once adjustment was made for the increase in false alarms. This suggests that training causes a bias for detecting the target emotion which generalises outside of the training task. However it does not increase accuracy for detecting the target emotion. The results are discussed in terms of the training’s utility in addressing different types of emotion processing deficits in psychological disorders.

Emotion recognition during cocaine intoxication

Chronic or repeated cocaine use has been linked to impairments in social skills. It is not clear whether cocaine is responsible for this impairment or whether other factors, like polydrug use, distort the observed relation. We aimed to investigate this relation by means of a placebo-controlled experimental study. Additionally, associations between stressor-related activity (cortisol, cardiovascular parameters) induced by the biological stressor cocaine, and potential cocaine effects on emotion recognition were studied.Twenty-four healthy recreational cocaine users participated in this placebo-controlled within-subject study. Participants were tested between 1 and 2h after treatment with oral cocaine (300mg) or placebo. Emotion recognition of low and high intensity expressions of basic emotions (fear, anger, disgust, sadness, and happiness) was tested.Findings show that cocaine impaired recognition of negative emotions; this was mediated by the intensity of the presented emotions. When high intensity expressions of Anger and Disgust were shown, performance under influence of cocaine ‘normalized’ to placebo-like levels while it made identification of Sadness more difficult. The normalization of performance was most notable for participants with the largest cortisol responses in the cocaine condition compared to placebo.It was demonstrated that cocaine impairs recognition of negative emotions, depending on the intensity of emotion expression and cortisol response.

Skilful communication: Emotional facial expressions recognition in very old adults

ObjectivesThe main objective of this study was to assess the changes associated with ageing in the ability to identify emotional facial expressions and to what extent such age-related changes depend on the intensity with which each basic emotion is manifested. MethodsA randomised controlled trial carried out on 107 subjects who performed a six alternative forced-choice emotional expressions identification task. The stimuli consisted of 270 virtual emotional faces expressing the six basic emotions (happiness, sadness, surprise, fear, anger and disgust) at three different levels of intensity (low, pronounced and maximum). The virtual faces were generated by facial surface changes, as described in the Facial Action Coding System (FACS). ResultsA progressive age-related decline in the ability to identify emotional facial expressions was detected. The ability to recognise the intensity of expressions was one of the most strongly impaired variables associated with age, although the valence of emotion was also poorly identified, particularly in terms of recognising negative emotions. ConclusionsNurses should be mindful of how ageing affects communication with older patients. In this study, very old adults displayed more difficulties in identifying emotional facial expressions, especially low intensity expressions and those associated with difficult emotions like disgust or fear.

Gaze Dynamics in the Recognition of Facial Expressions of Emotion

We studied preferably fixated parts and features of human face in the process of recognition of facial expressions of emotion. Photographs of facial expressions were used. Participants were to categorize these as basic emotions; during this process, eye movements were registered. It was found that variation in the intensity of an expression is mirrored in accuracy of emotion recognition; it was also reflected by several indices of oculomotor function: duration of inspection of certain areas of the face, its upper and bottom or right parts, right and left sides; location, number and duration of fixations, viewing trajectory. In particular, for low-intensity expressions, right side of the face was found to be attended predominantly (right-side dominance); the right-side dominance effect, was, however, absent for expressions of high intensity. For both low- and high-intensity expressions, upper face part was predominantly fixated, though with greater fixation of high-intensity expressions. The majority of trials (70%), in line with findings in previous studies, revealed a V-shaped pattern of inspection trajectory. No relationship, between accuracy of recognition of emotional expressions, was found, though, with either location and duration of fixations or pattern of gaze directedness in the face.

Sex differences in perception of invisible facial expressions.

Previous research indicates that women are better at recognizing facial expressions than men. In the current study, we examined whether this female advantage in the processing of facial expressions also occurs at the unconscious level. In two studies, participants performed a simple detection task and a 4-AFC task while faces were rendered invisible by continuous flash suppression. When faces with full intensity expressions were suppressed, there was no significant sex difference in the time of breakup of suppression (Study 1). However, when suppressed faces depicted low intensity expressions, suppression broke up earlier in men than women, indicating that men may be more sensitive to facial features related to mild facial expressions (Study 2). The current findings suggest that the female advantage in processing of facial expressions is absent in unconscious processing of emotional information. The female advantage in facial expression processing may require conscious perception of faces.

Multi-layer sparse representation for weighted LBP-patches based facial expression recognition.

In this paper, a novel facial expression recognition method based on sparse representation is proposed. Most contemporary facial expression recognition systems suffer from limited ability to handle image nuisances such as low resolution and noise. Especially for low intensity expression, most of the existing training methods have quite low recognition rates. Motivated by sparse representation, the problem can be solved by finding sparse coefficients of the test image by the whole training set. Deriving an effective facial representation from original face images is a vital step for successful facial expression recognition. We evaluate facial representation based on weighted local binary patterns, and Fisher separation criterion is used to calculate the weighs of patches. A multi-layer sparse representation framework is proposed for multi-intensity facial expression recognition, especially for low-intensity expressions and noisy expressions in reality, which is a critical problem but seldom addressed in the existing works. To this end, several experiments based on low-resolution and multi-intensity expressions are carried out. Promising results on publicly available databases demonstrate the potential of the proposed approach.

The visual discrimination of negative facial expressions by younger and older adults

Previous research has demonstrated that older adults are not as accurate as younger adults at perceiving negative emotions in facial expressions. These studies rely on emotion recognition tasks that involve choosing between many alternatives, creating the possibility that age differences emerge for cognitive rather than perceptual reasons. In the present study, an emotion discrimination task was used to investigate younger and older adults’ ability to visually discriminate between negative emotional facial expressions (anger, sadness, fear, and disgust) at low (40%) and high (80%) expressive intensity. Participants completed trials blocked by pairs of emotions. Discrimination ability was quantified from the participants’ responses using signal detection measures. In general, the results indicated that older adults had more difficulty discriminating between low intensity expressions of negative emotions than did younger adults. However, younger and older adults did not differ when discriminating between anger and sadness. These findings demonstrate that age differences in visual emotion discrimination emerge when signal detection measures are used but that these differences are not uniform and occur only in specific contexts.

Emotion perception across cultures: the role of cognitive mechanisms.

Despite consistently documented cultural differences in the perception of facial expressions of emotion, the role of culture in shaping cognitive mechanisms that are central to emotion perception has received relatively little attention in past research. We review recent developments in cross-cultural psychology that provide particular insights into the modulatory role of culture on cognitive mechanisms involved in interpretations of facial expressions of emotion through two distinct routes: display rules and cognitive styles. Investigations of emotion intensity perception have demonstrated that facial expressions with varying levels of intensity of positive affect are perceived and categorized differently across cultures. Specifically, recent findings indicating significant levels of differentiation between intensity levels of facial expressions among American participants, as well as deviations from clear categorization of high and low intensity expressions among Japanese and Russian participants, suggest that display rules shape mental representations of emotions, such as intensity levels of emotion prototypes. Furthermore, a series of recent studies using eye tracking as a proxy for overt attention during face perception have identified culture-specific cognitive styles, such as the propensity to attend to very specific features of the face. Together, these results suggest a cascade of cultural influences on cognitive mechanisms involved in interpretations of facial expressions of emotion, whereby cultures impart specific behavioral practices that shape the way individuals process information from the environment. These cultural influences lead to differences in cognitive styles due to culture-specific attentional biases and emotion prototypes, which partially account for the gradient of cultural agreements and disagreements obtained in past investigations of emotion perception.

Learning spatiotemporal features by using independent component analysis with application to facial expression recognition

Engineered features have been heavily employed in computer vision. Recently, feature learning from unlabeled data for improving the performance of a given vision task has received increasing attention in both machine learning and computer vision. In this paper, we present using unlabeled video data to learn spatiotemporal features for video classification tasks. Specifically, we employ independent component analysis (ICA) to learn spatiotemporal filters from natural videos, and then construct feature representations for the input videos in classification tasks based on the learned filters. We test the performance of proposed feature learning method with application to facial expression recognition. The experimental results on the well-known Cohn-Kanade database show that the learned features perform better than engineered features. The comparison experiments on recognition of low intensity expressions show that our method yields a better performance than spatiotemporal Gabor features.

Face Emotion Processing in Depressed Children and Adolescents with and without Comorbid Conduct Disorder

Studies of adults with depression point to characteristic neurocognitive deficits, including differences in processing facial expressions. Few studies have examined face processing in juvenile depression, or taken account of other comorbid disorders. Three groups were compared: depressed children and adolescents with conduct disorder (n = 23), depressed children and adolescents without conduct disorder (n = 29) and children and adolescents without disorder (n = 37). A novel face emotion processing experiment presented faces with 'happy', 'sad', 'angry', or 'fearful' expressions of varying emotional intensity using morphed stimuli. Those with depression showed no overall or specific deficits in facial expression recognition accuracy. Instead, they showed biases affecting processing of low-intensity expressions, more often perceiving these as sad. In contrast, non-depressed controls more often misperceived low intensity negative emotions as happy. There were no differences between depressed children and adolescents with and without conduct disorder, or between children with comorbid depression/conduct disorder and controls. Face emotion processing biases rather than deficits appear to distinguish depressed from non-depressed children and adolescents.