Facial Cues Research Articles

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition.

Mental illness has grown to become a prevalent and global health concern that affects individuals across various demographics. Timely detection and accurate diagnosis of mental disorders are crucial for effective treatment and support as late diagnosis could result in suicidal, harmful behaviors and ultimately death. To this end, the present study introduces a novel pipeline for the analysis of facial expressions, leveraging both the AffectNet and 2013 Facial Emotion Recognition (FER) datasets. Consequently, this research goes beyond traditional diagnostic methods by contributing a system capable of generating a comprehensive mental disorder dataset and concurrently predicting mental disorders based on facial emotional cues. Particularly, we introduce a hybrid architecture for mental disorder detection leveraging the state-of-the-art object detection algorithm, YOLOv8 to detect and classify visual cues associated with specific mental disorders. To achieve accurate predictions, an integrated learning architecture based on the fusion of Convolution Neural Networks (CNNs) and Visual Transformer (ViT) models is developed to form an ensemble classifier that predicts the presence of mental illness (e.g., depression, anxiety, and other mental disorder). The overall accuracy is improved to about 81% using the proposed ensemble technique. To ensure transparency and interpretability, we integrate techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) and saliency maps to highlight the regions in the input image that significantly contribute to the model's predictions thus providing healthcare professionals with a clear understanding of the features influencing the system's decisions thereby enhancing trust and more informed diagnostic process.

Identifying Children with Autism Spectrum Disorder via Transformer-Based Representation Learning from Dynamic Facial Cues

Identifying Children with Autism Spectrum Disorder via Transformer-Based Representation Learning from Dynamic Facial Cues

Drowsiness Detection System Through Eye and Mouth Analysis

Traffic jams are one of the serious issues in many developed countries. After the pandemic, many employees were allowed to travel interstate to work. This contributes to more severe jams, especially in the capital and nearby states. Long-distance driving and congestion can easily make the drivers sleepy and thus lead to traffic accidents. This paper aims to study and analyze facial cues to detect early symptoms of drowsy driving. The proposed method employs a deep learning approach, utilizing ensemble CNNs and Dlib's 68 landmark face detectors to analyze the facial cues. The analyzed symptoms include the frequency of eyes opened or closed and yawning or no yawning. Three individual CNN models and an ensemble CNN structure are built for the classification of the eyes and mouth yawn. The model training and validation accuracy graph and training loss and validation loss graph are plotted to verify that the models have not been overfitted. The ensemble CNN models achieved an approximate accuracy of 97.4% from the eyes and 96.5% from the mouth. It outperforms the other pre-trained models. The proposed system can immediately alert the driver and send text drowsy messages and emails to the third party, ensuring timely intervention to prevent accidents. The proposed method can be integrated into vehicles and transportation systems to ensure driver's safety. It can also be applied to monitor the driving behavior of those who drive long distances

Drowsiness Detection System Through Eye and Mouth Analysis

Traffic jams are one of the serious issues in many developed countries. After the pandemic, many employees were allowed to travel interstate to work. This contributes to more severe jams, especially in the capital and nearby states. Long-distance driving and congestion can easily make the drivers sleepy and thus lead to traffic accidents. This paper aims to study and analyze facial cues to detect early symptoms of drowsy driving. The proposed method employs a deep learning approach, utilizing ensemble CNNs and Dlib's 68 landmark face detectors to analyze the facial cues. The analyzed symptoms include the frequency of eyes opened or closed and yawning or no yawning. Three individual CNN models and an ensemble CNN structure are built for the classification of the eyes and mouth yawn. The model training and validation accuracy graph and training loss and validation loss graph are plotted to verify that the models have not been overfitted. The ensemble CNN models achieved an approximate accuracy of 97.4% from the eyes and 96.5% from the mouth. It outperforms the other pre-trained models. The proposed system can immediately alert the driver and send text drowsy messages and emails to the third party, ensuring timely intervention to prevent accidents. The proposed method can be integrated into vehicles and transportation systems to ensure driver's safety. It can also be applied to monitor the driving behavior of those who drive long distances

Drowsiness Detection System Through Eye and Mouth Analysis

Traffic jams are one of the serious issues in many developed countries. After the pandemic, many employees were allowed to travel interstate to work. This contributes to more severe jams, especially in the capital and nearby states. Long-distance driving and congestion can easily make the drivers sleepy and thus lead to traffic accidents. This paper aims to study and analyze facial cues to detect early symptoms of drowsy driving. The proposed method employs a deep learning approach, utilizing ensemble CNNs and Dlib's 68 landmark face detectors to analyze the facial cues. The analyzed symptoms include the frequency of eyes opened or closed and yawning or no yawning. Three individual CNN models and an ensemble CNN structure are built for the classification of the eyes and mouth yawn. The model training and validation accuracy graph and training loss and validation loss graph are plotted to verify that the models have not been overfitted. The ensemble CNN models achieved an approximate accuracy of 97.4% from the eyes and 96.5% from the mouth. It outperforms the other pre-trained models. The proposed system can immediately alert the driver and send text drowsy messages and emails to the third party, ensuring timely intervention to prevent accidents. The proposed method can be integrated into vehicles and transportation systems to ensure driver's safety. It can also be applied to monitor the driving behavior of those who drive long distances

Development of implicit and explicit attentional modulation of the processing of social cues conveyed by faces and bodies in children and adolescents.

Emotions and sex of other people shape the way we interact in social environments. The influence of these dimensions on cognitive processing is recognized as a highly conditional phenomenon. While much of researches on the topic focused on adults, less evidence is available for the pediatric population. This study aimed at examining the development of the modulation of attention control on emotion and sex processing using facial and body expressions in children and adolescents (8–16 years old). In Experiment 1a, participants performed a Flanker task (probing space-based attention) in which they had to indicate either the emotion (happy/fearful) or the sex of the target stimulus while ignoring the distracting stimuli at the side. We found evidence for intrusion of the sex, but not emotion, of the stimuli during both sex and emotion recognition tasks, thus both at an explicit (i.e., task relevant) and implicit (i.e., task irrelevant) level. A control experiment consisting of an emotional Flanker task confirmed that, in contrast with previous findings in adults, emotion did not modulate attention control in children and adolescents even when task relevant (Experiment 1b). In Experiment 2 participants performed a same-or-different judgment task (probing feature-based attention) in which they indicated whether the central stimulus matched the lateral ones for emotion or sex. Results showed that emotional features exerted an implicit influence during sex judgements; likewise, sex features intruded on the processing of both faces and bodies during emotion judgments. Finally, Experiment 3 explored the development of the explicit attention modulation exerted by the sex dimension on the processing of faces and bodies. To this aim, participants performed a Flanker task in which they were asked to recognize the sex of faces and bodies. The results indicated that, while younger participants showed a task-relevant influence of sexual features when processing faces, older participants showed such influence when processing bodies. These findings point to a greater attentional modulation exerted by sex, as compared to emotion, during social processing in children and adolescents and suggest a developmental trend of the saliency of facial and bodily cues for the perception of others’ sex.

FACING TRUSTWORTHINESS: AGE-GROUP DIFFERENCES IN TRUST-RELATED DECISION-MAKING AND LEARNING

Abstract Facial impressions contribute to evaluations of trustworthiness. Older adults are especially vulnerable to trust violations, incurring risks for deception and exploitation. Using the newly developed Social Iowa Gambling Task (S-IGT), we examined age-group differences in the impact of facial trustworthiness on decision-making. Advantageous decks were represented by trustworthy (congruent condition, CS-IGT) or untrustworthy (incongruent condition, IS-IGT) faces and disadvantageous decks were represented by untrustworthy faces in the CS-IGT and by trustworthy faces in the IS-IGT. Younger (n = 143) and older (n = 129) participants completed either the standard IGT, CS-IGT, or IS-IGT. Both age groups preferred trustworthy faces in their initial choices. Older adults performed worse than younger adults across all tasks over time. Further, compared to younger adults, older adults performed worse on the IS-IGT, suggesting that incongruent facial cues interfered with older adults’ performance, which aligns with reduced sensitivity to negative social reputations in aging. Exploratory multilevel modeling indicated that age-group differences were most pronounced in the last 40 trials across all tasks. Together these findings indicate that differences between younger and older adults in experience-dependent decision-making are magnified in social contexts that involve a “wolf in sheep’s clothing,” which may reflect age-related difficulties in information integration.

Nonverbal facial cues signaling sexually transmitted infections cause dehumanization and discrimination

Dehumanization often underlies the social ostracism, exclusion, and discrimination experienced by stigmatized group members. Given findings that people can detect sexually transmitted infection (STI) status from nonverbal facial cues, we tested whether people would dehumanize and discriminate against STI-positive individuals from detecting their stigmatized status. Specifically, we hypothesized that nonverbal stigma cues would stimulate dehumanizing reactions that lead to biases against hiring STI-positive individuals. Results showed that people dehumanize STI-positive individuals based on their nonverbal stigma cues (i.e., negative affect; Study 1), except when STI status is explicitly disclosed (Study 2), which leads to potential hiring biases (Study 3). Dehumanization and discrimination against STI-positive individuals may therefore depend on the stigma's legibility from nonverbal cues but may be tempered by explicit information about STI status.

Toddlers’ sensitivity to dominance traits from faces

In adults, seeing individual faces is sufficient to trigger dominance evaluations, even when conflict is absent. From early on, infants represent dyadic dominance relations and they can infer conflict outcomes based on a variety of cues. To date, it is unclear if toddlers also make automatic dominance trait evaluations of individual faces. Here we asked if toddlers are sensitive to dominance traits from faces, and whether their sensitivity depends on their face experience. We employed a visual preference paradigm to study 18- and 24-month-old toddlers’ sensitivity to dominance traits from three types of faces: artificial, male, female. When presented with artificial faces (Experiment 1), 18- and 24-month-olds attended longer to the non-dominant faces, but only when they were in upright orientation. For real male faces (Experiment 2), toddlers showed equivalent looking durations to the dominant and non-dominant upright faces. However, when looking at female faces (Experiment 3), toddlers displayed a visual preference for the upright non-dominant faces at 24 months. To our knowledge, this is the first study to show that toddlers already display sensitivity to facial cues of dominance from 18 months of age, at least for artificial face stimuli.

Multimodal aspects of sentence comprehension: Do facial and color cues interact with processing negated and affirmative sentences?

Negation is usually considered as a linguistic operator reversing the truth value of a proposition. However, there are various ways to express negation in a multimodal manner. It still remains an unresolved issue whether nonverbal expressions of negation can influence linguistic negation comprehension. Based on extensive evidence demonstrating that language comprehenders are able to instantly integrate extralinguistic information such as a speaker's identity, we expected that nonverbal cues of negation and affirmation might similarly affect sentence comprehension. In three preregistered experiments, we examined how far nonverbal markers of negation and affirmation-specifically, the so-called "not face" (see Benitez-Quiroz et al., 2016) and red or green color (see Dudschig et al., 2023)-interact with comprehending negation and affirmation at the sentential level. Participants were presented with photos ("not face" vs. positive control; Experiments 1 and 2) or color patches (red vs. green; Experiment 3). They then read negated and affirmative sentences in a self-paced manner or judged the sensibility of negated and affirmative sentences (e.g., "No, I do not want to sing" vs. "Yes, I would like to buy a sofa"). Both frequentist statistics and Bayes factors resulting from linear mixed-effects analyses showed that processing times for negated and affirmative sentences were not significantly modulated by the nonverbal features under investigation. This indicates that their influence might not extend to sentential negation or affirmation comprehension. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Male-biased sex ratios increase the support for a strong leader: An evolutionary threat management perspective

An evolutionary threat management approach to leadership assumes that humans have an adaptive followership psychology that responds to specific ecological threats by selecting appropriate leaders to mitigate any risks. We hypothesize that having an imbalanced sex ratio in society (with either an abundance of adult men or women) increases the support for a strong leader, because people find such imbalances threatening to themselves and to society. We tested our hypothesis with five preregistered experimental studies and one archival study. In the five online scenario studies (total N=2249), we manipulated the operational sex ratios within a fictitious country and measured both explicit leader preferences (ideal leader traits) and implicit preferences (facial cues of leaders). The results showed that a male-biased sex ratio (abundance of men) robustly increased the preference for a strong leader across all studies, but the effects of a female-biased sex ratio (abundance of women) were smaller and not consistent. Furthermore, a male-biased sex ratio in society activated the impression that society is a competitive, dangerous place. Overall, the studies enhance our understanding of the impact of an ecological factor, the sex ratio in society, on the support for political leadership.

Facial Stereotypes of Competence (Not Trustworthiness or Dominance) Most Resemble Facial Stereotypes of Group Membership

Previous research shows that perceivers have distinct mental representations of ingroups and outgroups even when groups are novel and not defined by physical attributes. Here, we leverage the minimal group paradigm, the reverse correlation method, and machine learning to parse the visual ingredients of group membership. In Study 1, we found that ingroup faces are trusted more than outgroup faces and that facial stereotypes of trustworthiness resemble those of the ingroup/outgroup distinction. However, in Study 2 we showed that such facial stereotypes of group membership resembled those of competence more than trustworthiness and dominance. Together, these findings suggest that even though trustworthiness is an important visual ingredient of the ingroup/outgroup distinction, people may rely on facial cues indicating competence the most to guide their visualization of novel ingroup and outgroup members, highlighting the nuanced nature of ingroup bias in face processing.

Gendered Perceptions of East and South Asian Men

Gendered race theory argues that in the United States, Asian men are perceived as more feminine and less masculine than Black and White men. However, extant research has focused predominantly on East Asians while overlooking South Asians. In five studies (N = 1,773 U.S. participants), we examined the perceived femininity-masculinity of East and South Asian men. East Asian men were perceived as more feminine and less masculine than South Asian, White, and Black men. South Asian men's perceived femininity-masculinity depends on whether they have facial hair or not. South Asian men with facial hair were perceived as less feminine, more masculine, and more threatening than men from other racial groups. This hyper-masculinization of South Asian men is likely due to alignment of the bearded terrorist stereotypes with facial hair cues. Results for South and East Asian women were inconsistent. This research highlights the importance of disaggregating Asian targets in social cognition theories.

The Usage of Artificial Intelligence Algorithms in Preventing Drunk Driving

Drunk driving is a very widespread problem, causing many casualties and millions of dollars in insurance and damages per year. In 2020, despite the COVID19 pandemic greatly reducing road traffic, about 32 people died per day in the US alone from DUI (Driving under Influence)(nhtsa.gov). Existing solutions such as police testing drivers suspected to be under influence is simply impractical considering the number of possible intoxicated drivers. With more and more people gaining access to a vehicle, it is crucial that more effective strategies be developed to detect and combat drunk driving. The purpose of this study is to analyze whether Artificial Intelligence can be used to more effectively detect and prevent drunk driving and if a Machine Learning models like Logistic Regression and Decision Tree can be more accurate than a human police officer. To address this question, a dataset named drunkImagesWebp was used. Two machine learning algorithms, Logistic regression and decision tree were trained on this dataset with facial image data of intoxicated people to predict the sobriety of humans based on facial cues. After testing this model, it is clear that both Logistic regression and decision tree models can accurately test a driver for signs of intoxication with well over 90% accuracy compared to human-administered tests which only hit up to around 75% accuracy. By comparison, both Logistic regression and Decision tree algorithms detected intoxication with 96% accuracy. This shows the potential AI can have in creating an automated solution to detecting and ultimately preventing drunk driving.

Blindness influences emotional authenticity perception in voices: Behavioral and ERP evidence

The ability to distinguish spontaneous from volitional emotional expressions is an important social skill. How do blind individuals perceive emotional authenticity? Unlike sighted individuals, they cannot rely on facial and body language cues, relying instead on vocal cues alone. Here, we combined behavioral and ERP measures to investigate authenticity perception in laughter and crying in individuals with early- or late-blindness onset. Early-blind, late-blind, and sighted control participants (n = 17 per group, N = 51) completed authenticity and emotion discrimination tasks while EEG data were recorded. The stimuli consisted of laughs and cries that were either spontaneous or volitional. The ERP analysis focused on the N1, P2, and late positive potential (LPP). Behaviorally, early-blind participants showed intact authenticity perception, but late-blind participants performed worse than controls. There were no group differences in the emotion discrimination task. In brain responses, all groups were sensitive to laughter authenticity at the P2 stage, and to crying authenticity at the early LPP stage. Nevertheless, only early-blind participants were sensitive to crying authenticity at the N1 and middle LPP stages, and to laughter authenticity at the early LPP stage. Furthermore, early-blind and sighted participants were more sensitive than late-blind ones to crying authenticity at the P2 and late LPP stages.Altogether, these findings suggest that early blindness relates to facilitated brain processing of authenticity in voices, both at early sensory and late cognitive-evaluative stages. Late-onset blindness, in contrast, relates to decreased sensitivity to authenticity at behavioral and brain levels.

Walk as you feel: Privacy preserving emotion recognition from gait patterns

Emotion recognition from gait has gained significant interest due to its applicability in different fields such as healthcare, social cues, surveillance, and smart applications. Gait, as a biometric trait, offers unique advantages, allowing remote identification and robust recognition even in uncontrolled scenarios. Moreover, gait analysis can provide valuable insights into an individual’s emotional state. This work presents the “Walk-as-you-Feel” (WayF) framework, a novel approach for gait-based emotion recognition that does not rely on facial cues, ensuring user privacy. To address challenges with small and unbalanced datasets, a balancing procedure suitable for deep learning architecture is also developed. Adapted Inception-v3 and EfficientNet are employed for the feature extraction phase. Classification is performed using a Gated Recurrent Units network (GRUs) and Transformers-Encoder. Experimental results demonstrate the competitiveness of the proposed approach with respect to state-of-the-art works which also integrate facial cues. WayF reaches an average recognition rate of approximately 77% in its best configuration. Moreover, when excluding the neutral emotion, the proposed method achieves an outstanding overall accuracy of 83.3%.

Social processing modulates the initial allocation ofattention towards angry faces: evidence from the N2pc component.

Previous research has shown that attentional bias towards angry faces is moderated by the activation of a social processing mode. More specifically, reliable cueing effects for angry face cues in the dot-probe task only occurred when participants performed a task that required social processing of the target stimuli. However, cueing effects are a rather distal measure of covert shifts in spatial attention. Thus, it remains unclear whether the social processing mode modulates initial allocation of attention to or attentional disengagement from angry faces. In the present study, we used the N2pc, an event-related potential component, as an index of attentional shifts towards angry faces. Participants performed a dot-probe task with two different target conditions while the electroencephalogram (EEG) was recorded. In the social target condition, target stimuli were socially meaningful (schematic faces), and in the non-social target condition, they were meaningless (scrambled schematic faces). The amplitude of the N2pc component elicited by angry face cues was significantly larger in the social target condition than in the non-social target condition. This pattern also occurred for behavioural cueing effects. These results suggest that the activation of a social processing mode due to current task demands affects the initial allocation of attention towards angry faces.

We cannot find peace in restlessness. How the neuroscience supports psychotherapeutic practice

In this article, I explore how some of the more recent developments in the neuroscience based understanding of human brain and body functioning both illuminates and underpins aspects of psychotherapeutic practice. Psychotherapists have accumulated much wisdom about people and their relationships, based on observation and fine-tuned feedback from therapy practice. Psychotherapists have always known that relational stress and high levels of arousal impact intimate communication in difficult relational moments – it slows information processing, makes it harder to read facial cues and makes it more likely we become preoccupied with our own affective state, along with a self-defensive impediment to listening. Our increased knowledge of these brain and body states helps to guide the therapist and client to effective action. It is my contention that systemic training needs to include human physiology and anatomy as a core part of learning.

Canine Curiosity: What We Do and Don’t Know, and What Human Infants Could Teach Us

The phenomenon of domesticated dogs looking to humans for information is ubiquitous, yet infrequently observed among other interspecies interactions. Dogs’ inclination to solicit information from humans is in large part a result of the two species’ shared social evolution and niche. Perhaps a more compelling aspect of this relationship is how dogs respond in the face of unexpected, uncertain, and/or novel cues from humans, from whom they frequently solicit information. The influence of human presence on canines’ curiosity about and engagement with their immediate environment is understudied, in part due to challenges in study design. SomeS of these challenges are common to working with and learning from babies of our own species. And, as dogs have developed many mental processes and behaviors similar to preverbal human infants, illuminating strategies for understanding curiosity in babies may prove useful in learning more about how dogs experience the world, with and without people.

Neural discrimination of facial cues associated with trustworthiness in adults and 6-month-old infants as revealed by fast periodic visual stimulation.

Adults and children easily distinguish between fine-grained variations in trustworthiness intensity based on facial appearance, but the developmental origins of this fundamental social skill are still debated. Using a fast periodic visual stimulation (FPVS) oddball paradigm coupled with electroencephalographic (EEG) recording, we investigated neural discrimination of morphed faces that adults perceive as low- and high-trustworthy in a sample of 6-month-old infants (N = 29; 56% male; Mage = 196.8 days; all White) and young adults (N = 21; 40% male; Mage = 24.61 years; all White) recruited in Italy. Stimulus sequences were presented at 6 Hz with deviant faces interleaved every fifth stimulus (i.e., 1.2 Hz); oddball category (high/low trustworthiness) was varied within subjects. FPVS responses were analyzed at both frequencies of interest and their harmonics as a function of deviant type (high- vs. low-trustworthy) over occipital and occipitolateral electrode clusters. For both infants and adults, the baseline response did not differ between trustworthiness conditions. Significant responses were centered on the right parietal electrodes in infants, and on the occipital and left occipitotemporal clusters in adults. Oddball responses were significant for both infants and adults, with cross-age differences in the topographical localization of the response on the scalp. Overall, results suggest that, by the age of 6 months, infants discriminate between faces that adults rate as high and low in trustworthiness, extending prior evidence of early sensitivity to this face dimension in humans. (PsycInfo Database Record (c) 2023 APA, all rights reserved).