Emotional Facial Action Coding System Research Articles

Machine learning based model to predict Human Emotions using facial features

This paper focuses on Emotion Recognition using facial expressions. It aims to detect facial expressions accurately and efficiently. Here, we have used facial expressions as the criteria wherein the lips, eyebrow movement, eyes, etc. are the factors that help us detect various emotions. The purpose of this paper is to serve various fields where emotion recognition will be viable and can be helpful for better outcomes. Spectrums like video gaming, psychology, medical fields, etc. can use Emotion Recognition System in a very positive way. Convolutional Neural Network has developed to help in recognizing emotions through facial expressions and classify them into seven basic categories which are: happy, sad, neutral, surprise, fear, disgust, and angry. So, Convolutional Neural Network is implemented to extract relevant features of the input images and classify them into seven categories. For evaluating the proposed model, Facial Emotion Recognition 2013 dataset is used so that the model achieves the best accuracy rate. Keywords—Convolutional Neural Network (CNN), Facial Emotion Recognition 2013 (FER 2013), Extended Cohn-Kanade (CK+), Japanese Female Facial Expressions (JAFFE), Emotional Facial Action Coding System (EMFACs), Action Units (AUs).

Facial Affective Behavior in Borderline Personality Disorder Indicating Two Different Clusters and Their Influence on Inpatient Treatment Outcome: A Preliminary Study.

BackgroundThe purpose of the present study was threefold: first, to investigate the facial affective behavior in patients with a borderline personality disorder (BPD); second, to examine whether these patients could be divided into clusters according to facial affective behavior; and third, to test whether these clusters would influence the inpatient treatment outcome.MethodsThirty inpatients with BPD were assessed with the Structured Clinical Interviews for DSM-IV Axis I and II Disorders (SCID I, SCID II) and had to complete a series of questionnaires before and directly after the 12-week long inpatient treatment. Facial affective behavior was recorded during the structured interview for personality organization (STIPO) and afterward coded with the emotional facial action coding system (EMFACS). Measures on psychopathology [beck depression inventory (BDI), Spielberger state and trait anxiety inventory (STAI), Spielberger state and trait anger inventory (STAXI), and symptom cheklist-90-revised (SCL-90-R)], interpersonal problems [Inventory of Interpersonal Problems (IIP)], and personality organization [inventory of personality organization (IPO)] were administered.ResultsCluster analysis before the treatment yielded two groups that differed in general facial expressivity, and regarding the display of anger, contempt, and disgust. The effect sizes of the repeated measures ANOVAs showed that persons with higher scores on the affective facial expressions benefitted more from the treatment in terms of STAI state anxiety, STAXI state and trait anger, IIP total, and the two scales primitive defenses and identity diffusion of the IPO, whereas persons with lower scores benefitted more on the scale IPO reality testing.ConclusionOur results indicated some initial trends for the importance of facial affective behavior in patients with BPD and their treatment outcome.

Interpretation and Working through Contemptuous Facial Micro-Expressions Benefits the Patient-Therapist Relationship.

Introduction: The significance of psychotherapeutic micro-processes, such as nonverbal facial expressions and relationship quality, is widely known, yet hitherto has not been investigated satisfactorily. In this exploratory study, we aim to examine the occurrence of micro-processes during psychotherapeutic treatment sessions, specifically facial micro-expressions, in order to shed light on their impact on psychotherapeutic interactions and patient-clinician relationships. Methods: In analyzing 22 video recordings of psychiatric interviews in a routine/acute psychiatric care unit of Vienna General Hospital, we were able to investigate clinicians’ and patients’ facial micro-expressions in conjunction with verbal interactions and types. To this end, we employed the Emotion Facial Action Coding System (EmFACS)—assessing the action units and microexpressions—and the Psychodynamic Intervention List (PIL). Also, the Working Alliance Inventory (WAI), assessed after each session by both patients and clinicians, provided information on the subjective quality of the clinician–patient relationship. Results: We found that interpretative/confrontative interventions are associated with displays of contempt from both therapists and patients. Interestingly, displays of contempt also correlated with higher WAI scores. We propose that these seemingly contradictory results may be a consequence of the complexity of affects and the interplay of primary and secondary emotions with intervention type. Conclusion: Interpretation, confrontation, and working through contemptuous microexpressions are major elements to the adequate control major pathoplastic elements. Affect-cognitive interplay is an important mediator in the working alliance.

A visual attention based ROI detection method for facial expression recognition

Abstract In this paper, an eleven-layered Convolutional Neural Network with Visual Attention is proposed for facial expression recognition. The network is composed of three components. First, local convolutional features of faces are extracted by a stack of ten convolutional layers. Second, the regions of interest are automatically determined according to these local features by the embedded attention model. Third, the local features in these regions are aggregated and used to infer the emotional label. These three components are integrated into a single network which can be trained in an end-to-end scheme. Extensive experiments on four kinds of data (namely aligned frontal faces, faces in different poses, aligned unconstrained faces, and grouped unconstrained faces) prove that the proposed method can improve the accuracy and obtain good visualization. The visualization shows that the learned regions of interest are partly consistent with the locations of emotion specific Action Units. This founding confirms the interpretation of Facial Action Coding System and Emotional Facial Action Coding System from a machine learning perspective.

Diversity from within: The Impact of Cultural Variables on Emotion Expressivity in Singapore

<p>Culture is intrinsically linked with emotion expression, as culture provides rules regarding how to manage emotions when they occur. Thus far, existing literature has extensively compared norms for emotional expression and suppression, revealing significant differences among culturally distinct but also geographically distant groups (e.g., “collectivistic” Chinese versus “individualistic” U.S. Americans). The present study examines the impact of cultural diversity within Singapore, a heterogeneous Asian nation of 5.4 million residents. Using an expression suppression paradigm, eighty-three participants viewed emotion eliciting video clips and their expressions were analyzed according to the Emotion Facial Action Coding System (EmFACS, Ekman, Irwin, & Rosenberg, 1994) for signs of happiness and disgust. Participants tasked to manage their expression were successful; however cultural indicators such as ethnicity, collectivism, and concern for face affected expressivity under both suppression and natural expression conditions. These results emphasize the importance of exploring culture within national boundaries, as multiple cultural factors (e.g., ethnic groupings, values, and face) influenced expression.</p>

Facial emotional expression in schizophrenia adolescents during verbal interaction with a parent

Research on emotional expression in adult schizophrenia patients has indicated an impairment of the patients' facial expressions during social interaction. However, it is unclear whether schizophrenia adolescent-onset patients show comparable disturbances in facial expression. Our aim was to analyze facial emotions in adolescents with schizophrenia spectrum psychoses compared with non-patient controls during a problem-solving discussion with their primary caregivers. We assessed facial expressions of emotions and speaker position (speaking vs. listening) in schizophrenia and non-patient adolescents matched for age and gender. The facial emotions were coded with the Emotional Facial Action Coding System. Schizophrenia adolescents showed significantly lower rates per minute of expressions indicating positive emotions compared with non-patient adolescents (β = -0.95, 95 % CI [-1.34, -0.57], t (37.92) = -5.00, p ≤ 0.001). In contrast, there was no lower rate of expressions indicating negative emotions of schizophrenia adolescents (β = 0.10, CI [-0.42, 0.61], t (35.03) = 0.38, p = 0.709). While the negative facial expressions of non-patient adolescents were related to speaking (β = 0.63, CI [0.34, 0.92], t (26) = 4.50, p = < 0.001), this relationship was not observed in schizophrenia adolescents. Our results indicated an emotion-specific impairment of positive facial expressions in schizophrenia adolescents early in the course of illness and a deviant relation between facial emotional expressions and speech.

Stability of Facial Affective Expressions in Schizophrenia

Thirty-two videorecorded interviews were conducted by two interviewers with eight patients diagnosed with schizophrenia. Each patient was interviewed four times: three weekly interviews by the first interviewer and one additional interview by the second interviewer. 64 selected sequences where the patients were speaking about psychotic experiences were scored for facial affective behaviour with Emotion Facial Action Coding System (EMFACS). In accordance with previous research, the results show that patients diagnosed with schizophrenia express negative facial affectivity. Facial affective behaviour seems not to be dependent on temporality, since within-subjects ANOVA revealed no substantial changes in the amount of affects displayed across the weekly interview occasions. Whereas previous findings found contempt to be the most frequent affect in patients, in the present material disgust was as common, but depended on the interviewer. The results suggest that facial affectivity in these patients is primarily dominated by the negative emotions of disgust and, to a lesser extent, contempt and implies that this seems to be a fairly stable feature.

Facial emotional expression in reaction to social exclusion in borderline personality disorder

Disturbances in social interaction are a defining feature of patients with borderline personality disorder (BPD). In this study, facial emotional expressions, which are crucial for adaptive interactions in social contexts, were assessed in patients with BPD in response to social exclusion. We examined facial emotional reactions of 35 patients with BPD and 33 healthy controls when playing Cyberball, a virtual ball-tossing game that reliably induces social exclusion. Besides self-reported emotional responses, facial emotional expressions were analyzed by applying the Emotional Facial Action Coding System (EMFACS). Patients with BPD showed a biased perception of participation. They more readily reported feeling excluded compared to controls even when they were included. In BPD, social exclusion led to an increase in self-reported other-focused negative emotions. Overall, EMFACS analyses revealed that BPD patients reacted with fewer positive expressions and with significantly more mixed emotional expressions (two emotional facial expressions at the same time) compared to the healthy control group when excluded. Besides a negative bias for perceived social participation, ambiguous facial emotional expressions may play an important role in the disturbed relatedness in patients with BPD.

Crime Scene Investigation: The Chief Inspectors’ Display Rules

Television as one factor in the “cycle of role division” (Eagly 1987; Shields 2002) can aid to perpetuate traditional role attitudes as well as set new norms. The aim of the study is to examine gender differences in facial expression of emotion on TV as well as changes of these differences over time. A content analysis of 12 episodes of a prominent German crime series from two time samples (1979–1981 / 2005) was done by employing the Emotional Facial Action Coding System (EmFACS, Friesen and Ekman 1983). Results show that there are gender differences in stereotypically male emotions: Male actors express these emotions significantly more frequently than female actors, while the latter show more “social smiles”. These differences are reduced over time.

Dyadic Facial Affective Indicators of Severity of Symptomatic Burden in Patients with Panic Disorder

Background: Based on the results of research on facial affective behaviour in different psychological disorders, as well as on available findings on the specific behaviour of patients with panic disorder in interaction with their therapists, hypotheses about dyadic facial affective behaviour and its correlation with symptomatic burden of female panic patients are formulated. Sampling and Methods: The facial affective behaviour of 20 patients with panic disorder and their therapists, coded with the Emotional Facial Action Coding System, in the first treatment session is analyzed regarding interactive enmeshment, and for a subgroup of 15 dyads these data are correlated with those on symptomatic burden before treatment. Results: A high degree of interactive enmeshment between patient and therapist correlates positively with the severity of symptomatic burden. All dyadic enmeshment indicators show highly significant positive correlations with body-related symptoms, but not with more general variables like global severity index of the SCL-90R or general anxiety (State Trait Anxiety Inventory). Conclusions: These results are discussed against the background of specific psychodynamics of panic patients and show that, on the one hand, therapists practise an interactive abstinence, but on the other hand, they tend to be pulled into a specific interactive enmeshment by patients with greater symptomatic burden. Limitations of the study arise from the small sample and the lack of a comparison group, therefore the question if the results are disorder specific or more general cannot be answered.

Mimisch-affektives Verhalten bei Patientinnen mit Angststörungen während des Adult-Attachment-Interviews: eine Pilotstudie

In this study we examined for the first time the difference between patients with an anxiety disorder and healthy controls in their attachment representation and facial affective behavior during the activation of the attachment system. 13 female patients und 14 healthy women were administered with the Adult Attachment Interview (AAI). Facial affective behavior during 6 selected questions of the AAI was coded using the Emotional-Facial-Action-Coding-System (EMFACS). As expected patients with an anxiety disorder, especially panic disorders, were classified significantly more often as insecure-preoccupied with a high proportion of unresolved loss. Against our assumption anxiety patients, independent of their attachment category, did not differ in their facial affective behavior from the control group. A group comparison taking into account diagnosis and attachment status showed that duchenne smile (happiness) was significantly predominant in control subjects classified as secure. Attachment security in healthy subjects, characterized by an overall valuing of positive or negative attachment experiences and coherent discourse in the AAI, was associated with positive facial affectivity. In contrast insecure anxiety patients could be characterized by showing social smile when talking e. g. about former separation experiences from their attachment figures mostly in an incoherent manner. This could be interpreted as a self-regulating defense. Limitations of the study are the small sample size and the heterogeneous clinical group of anxiety disorders.

The prototypical pride expression: Development of a nonverbal behavior coding system.

This research provides a systematic analysis of the nonverbal expression of pride. Study 1 manipulated behavioral movements relevant to pride (e.g., expanded posture and head tilt) to identify the most prototypical pride expression and determine the specific components that are necessary and sufficient for reliable recognition. Studies 2 and 3 tested whether the 2 conceptually and empirically distinct facets of pride ("authentic" and "hubristic"; J. L. Tracy & R. W. Robins, 2007a) are associated with distinct nonverbal expressions. Results showed that neither the prototypical pride expression nor several recognizable variants were differentially associated with either facet, suggesting that for the most part, authentic and hubristic pride share the same signal. Together these studies indicate that pride can be reliably assessed from nonverbal behaviors. In the Appendix, the authors provide guidelines for a pride behavioral coding scheme, akin to the Emotion Facial Action Coding System (EMFACS; P. Ekman & E. Rosenberg, 1997) for assessing "basic" emotions from observable nonverbal behaviors.

Facial expression of emotions in borderline personality disorder and depression

Borderline personality disorder (BPD) is characterized by marked problems in interpersonal relationships and emotion regulation. The assumption of emotional hyper-reactivity in BPD is tested regarding the facial expression of emotions, an aspect highly relevant for communication processes and a central feature of emotion regulation. Facial expressions of emotions are examined in a group of 30 female inpatients with BPD, 27 women with major depression and 30 non-patient female controls. Participants were videotaped while watching two short movie sequences, inducing either positive or negative emotions. Frequency of emotional facial expressions and intensity of happiness expressions were examined, using the Emotional Facial Action Coding System (EMFACS-7, Friesen & Ekman, EMFACS-7: Emotional Facial Action Coding System, Version 7. Unpublished manual, 1984). Group differences were analyzed for the negative and the positive mood-induction procedure separately. Results indicate that BPD patients reacted similar to depressed patients with reduced facial expressiveness to both films. The highest emotional facial activity to both films and most intense happiness expressions were displayed by the non-clinical control group. Current findings contradict the assumption of a general hyper-reactivity to emotional stimuli in patients with BPD.

Alexithymic Characteristics and Patient-Therapist Interaction: A Video Analysis of Facial Affect Display

Alexithymia as a disorder of affect regulation entails a patient’s reduced ability to process emotional information. The purpose of this study was to evaluate the impact of alexithymia [as measured by the Toronto Alexithymia Scale (TAS)-26, German version] on affective correlates in a dyadic therapeutic interaction (as recorded by the Emotional Facial Action Coding System). Interviews with 12 in-patients with various psychosomatic disorders (anxiety, depression, somatisation) were videotaped and evaluated for facial affect display. The corresponding emotional reactions of the therapists (split screen) were recorded separately. Patients with high alexithymia scores (TAS-26 total score) tended to display less aggressive affects than those with low scores. The therapists’ predominant emotional reaction to alexithymic patients was contempt. Our findings underscore the deep-rooted nature of alexithymia as a disorder of affect regulation. Since facial affects play a major role in the regulation of emotional interaction, this disorder may evoke negative reactions of potential caregivers.

Intravenous Sedation for Control of Distress during Lumbar Punctures for Pediatric Cancer Patients

OBJECTIVE: To compare the sedative and amnestic effects, the rapidity of recovery, and the adverse effects of midazolam and propofol for intravenous sedation. The focus of this paper is the effects of intravenous sedation on distress, the acceptability to parents of this intervention for helping their children cope with painful procedures and the utility of an intravenous sedation program administered by an anesthetist.PATIENTS AND METHODS: Twenty six children were randomly assigned to intravenous sedation with either midazolam or propofol as the first drug received. Self-report measures of anxiety and fear were assessed before and 30 min after the lumbar puncture (LP). Self-report of pain was also assessed 30 min post-LP. The children were videotaped during the LP, and the tapes were coded for pain and negative emotion using the Emotion Facial Action Coding System. One parent completed the acceptability questionnaire while the child was in the recovery room.RESULTS: Only three children indicated pain on the Visual Analogue Scale, and two children had facial expressions indicative of pain during the LP. Fear and anxiety differed as a function of procedure order, and children were more fearful and anxious before the first sedated LP than the second. Intravenous sedation was highly acceptable to parents, and neither drug was considered superior.CONCLUSIONS: Intravenous sedation with either propofol or midazolam, in combination with analgesia, was effective for alleviating pain and emotional distress associated with LPs. Although children were told that the sedation would make them relaxed and sleepy, they remained somewhat fearful until they experienced a comfortable LP. The benefits of an anesthetist-administered intravenous sedation program are discussed.

Emotional Experience and Facial Behavior During the Psychotherapeutic Process and its Relation to Treatment Outcome: A Pilot Study

This study examined emotionality and affective exchange during psychotherapeutic treatments by investigating how these processes are related to each other and to treatment outcome. Facial expressions of emotion and its temporal sequence were described using the Emotional Facial Action Coding System (EMFACS) for two videotaped brief psychoanalytic treatments, a successful one and a therapeutic failure. Patient and therapist affective experience were measured with the Differentielle Affekt-Skala (DAS). High correlations were found between facial expressions of emotion and affective experience. Some correlations were the same in both dyads, but most were dyad-specific. Furthermore, aspects of facial behavior and the interaction partner's responses were both good predictors of affective experience. Interpretation of the meaning of facial behavior and the measurement of affective experience provides insights into the nature of the psychotherapeutic process and its relation to success and failure. The two treat...

Preschoolers' moral actions and emotions in Prisoner's Dilemma.

The moral acts of 19 dyads of 4-year-olds in a cognitively simplified version of Prisoner's Dilemma were analyzed in relationship to their friendship, emotions, and processes of conflict resolution. Degree of friendship was rated by teachers; two sociologists used a Q-sort of Group Processes to describe the dyads' interactions; the emotions of each child were coded from videotape--independently of his or her partner's and with the sound turned off--in accordance with Ekman and Friesen's (1983) Emotion Facial Action Coding System. Moral acts were categorically identified as equalization, reparation, stalemate, default, and betrayal; these acts were also assigned scale scores on a dimension of moral sensitivity that was independently derived from separate work with a sample of 143 undergraduates. They judged the five acts in all possible paired comparisons and then rated the moral difference between each pair. These data almost perfectly fitted a model of increasing monotonic function according to Kruskal's (1964) criterion. The degree of friendship between the dyads and their interactive processes---group orientation, positive emotional tone, and active involvement--were positively related to sensitive moral action. Their emotions were related to concurrent acts, and more importantly, predicted their subsequent acts even though they seldom talked about morality. These results suggest that if social scientists' search for practical morality is to be successful, emotional communication needs to be brought into the account. A substantial and still growing number of investigations (for a few of the many examples, see Main & George, 1985; Rheingold & Hay, 1980, Strayer, 1980; and an extensive review by Radke-Yarrow, Zahn-Waxler, & Chapman, 1983) suggest that young children may be neither as morally insensitive nor incapable as held in theories that define morality and its development as primarily due to cognitive development or socialization. These contrary findings especially seem to occur in investigations that concern young children's moral actions rather than their judgments about hypothetical dilemmas. Still those cognitive scientists (for instance, Anderson, 1980) who have taken special care in simplifying and clarifying the cognitive requirements of their tasks also report such findings. Thus this growing body of work raises questions about the nature of young children's morality. Here we are concerned only with the conditions of their action, which is surely the more convincing test of moral capability. An obvious but not frequently studied feature of moral action is its heat--its hot in contrast to the cool cognitions evoked by hypothetical problems