Asymmetric Facial Expressions Research Articles

CSINet: Channel–Spatial Fusion Networks for Asymmetric Facial Expression Recognition

Occlusion or posture change of the face in natural scenes has typical asymmetry; however, an asymmetric face plays a key part in the lack of information available for facial expression recognition. To solve the problem of low accuracy of asymmetric facial expression recognition, this paper proposes a fusion of channel global features and a spatial local information expression recognition network called the “Channel–Spatial Integration Network” (CSINet). First, to extract the underlying detail information and deepen the network, the attention residual module with a redundant information filtering function is designed, and the backbone feature-extraction network is constituted by module stacking. Second, considering the loss of information in the local key area of face occlusion, the channel–spatial fusion structure is constructed, and the channel features and spatial features are combined to enhance the accuracy of occluded facial recognition. Finally, before the full connection layer, more local spatial information is embedded into the global channel information to capture the relationship between different channel–spatial targets, which improves the accuracy of feature expression. Experimental results on the natural scene facial expression data sets RAF-DB and FERPlus show that the recognition accuracies of the modeling approach proposed in this paper are 89.67% and 90.83%, which are 13.24% and 11.52% higher than that of the baseline network ResNet50, respectively. Compared with the latest facial expression recognition methods such as CVT, PACVT, etc., the method in this paper obtains better evaluation results of masked facial expression recognition, which provides certain theoretical and technical references for daily facial emotion analysis and human–computer interaction applications.

AFLFP: A Database With Annotated Facial Landmarks for Facial Palsy

Facial landmark detection is a crucial step for the task of computer-aided facial palsy diagnosis, which enables to focus on the affected facial regions for learning asymmetry, shape, and texture features of facial palsy. However, it is still very challenging to accurately detect salient landmarks on facial palsy images due to the unavailability of sufficient training databases providing annotated facial landmark images of facial palsy. To this end, we present a database in this article named annotated facial landmarks for facial palsy (AFLFP). AFLFP is a diverse, and reliable database that contains facial images with 16-class facial expressions of asymmetric facial expressions from 88 subjects. Each facial image is independently and manually annotated with 68 facial landmarks. This database is the first public manually annotated facial landmark database for facial palsy so far. Furthermore, to establish the benchmark results for the proposed database, we propose a deep neural network (DNN) baseline with a two-stage cascaded fully convolutional network (FCN), which can detect facial landmarks in facial palsy from coarse to fine. The comprehensive experiments show that the proposed method performs better than the mainstream methods of machine-and deep-learning. And we have also compared the performance using normal-and palsy-faces, respectively, as the training data. The comparison results show that there are significant differences between them in terms of facial landmark detection, which further confirms the necessity to develop a facial landmark database specifically for facial palsy.

Consumer responses toward symmetric versus asymmetric facial expression emojis

Facial expression emojis are commonly used in digital communications and marketing campaigns. However, it is less known how the design of emojis may affect consumer responses. To address this gap, this research examines the impact of emojis’ facial asymmetry levels on consumer judgments. Findings across four studies demonstrate that compared with their symmetric counterparts, asymmetric facial expression emojis are more likely to receive favorable consumer evaluations. This effect is driven by perceptions of human expression resemblance and emotional expression strength and tends to be more prominent among consumers with a higher level of emotional sensitivity. Moreover, marketing messages including an asymmetric (vs. a symmetric) emoji are more likely to generate positive consumer responses.

Multifaceted rehabilitation strategies for ramsay hunt syndrome: a case report and a review of literature

Background: Ramsay Hunt Syndrome (RHS) is a scarce diagnosis involving unilateral facial paralysis resulting from the reactivation of Varicella-Zoster Virus (VZV) infection in the past, which causes pain, asymmetrical facial expressions, and difficulty in speaking, eating and drinking. The most effective treatment is still debatable, and limited research is available. Rehabilitation program is considered as the core treatment that gives functional improvement. The aim of this case study is to present clinical findings in RHS and the rehabilitation program to maximize facial expression recovery.Case Description: A 59-year-old man came to rehabilitation medicine outpatient clinic two weeks after diagnosed as RHS, with complaints of fever; severe pain and rashes on the left side of the face; and ear discomfort on the same side. Physical examination revealed peripheral facial nerve palsy on the left side with drooping of the left eyelid, left end of the mouth, and difficulty in oral communication. Electroneuromyography (ENMG) examination showed axonal facial nerve paralysis on the left side. The patient was referred to the physiatrist and got rehabilitation program with neuromuscular electrical stimulation (NMES), facial massage and biofeedback exercise for facial muscle 3 times a week for 4 weeks.Discussion: After 4 weeks rehabilitation program, the patient showed improvement as House-Brackmann grades improved from a grade IV to a grade II; improvements in communication, facial symmetry at both rest and motion; significant improvement in experiencing pain. Conclusions: This study suggested that rehabilitation program with NMES, facial massage and biofeedback exercise are safe, efficacious and provide good outcomes in the treatment of Ramsay Hunt Syndrome.

Abstract TP304: Mimicking Acute Stroke Findings With a Digital Avatar

Introduction: It is now well-accepted that simulation-based learning (SBL) is an important component of medical education. At our institution, we have a state-of-the-art simulation center, and SBL is already incorporated into many medical subspecialties. However, commercially available patient simulators have static faces and lack the realistic depiction of non-verbal facial cues important for rapid diagnosis of neurological emergencies such as stroke. This multidisciplinary project addresses the urgent need for expressive patient simulators by developing acute stroke avatars for use in simulated healthcare training. Methods: Using a previously published and validated method, we developed techniques to display stroke pathologies on a virtual patient simulator (VPS). After obtaining patient or surrogate consent, we obtained source videos from patients admitted to an academic medical center who had experienced acute ischemic stroke resulting in neurological findings such as facial droop, eyelid apraxia, dysarthria, and coma. We then extracted facial features using shape-based modeling techniques, leaving anonymous feature points. Next, we applied a novel algorithm to use these feature points to build accurate computational models (masks) representing the facial characteristics of stroke. We then overlaid these prebuilt masks onto a live facial video stream to generate asymmetric expressions on a virtual avatar. This project was IRB approved. Results: More than 21 videos of stroke patients were made. Once the feature points were extracted from these videos, we were able to develop VPSs capable of expressing realistic asymmetric facial expressions. These avatars were then validated amongst neurologists with clinical experience in diagnosing acute ischemic stroke. Conclusions: This multidisciplinary effort using patient-inspired facial expressions resulted in a tool that aids the stroke education community by making virtual and robotic simulators of acute stroke more varied, interactive, and realistic.

Primate modularity and evolution: first anatomical network analysis of primate head and neck musculoskeletal system

Network theory is increasingly being used to study morphological modularity and integration. Anatomical network analysis (AnNA) is a framework for quantitatively characterizing the topological organization of anatomical structures and providing an operational way to compare structural integration and modularity. Here we apply AnNA for the first time to study the macroevolution of the musculoskeletal system of the head and neck in primates and their closest living relatives, paying special attention to the evolution of structures associated with facial and vocal communication. We show that well-defined left and right facial modules are plesiomorphic for primates, while anthropoids consistently have asymmetrical facial modules that include structures of both sides, a change likely related to the ability to display more complex, asymmetrical facial expressions. However, no clear trends in network organization were found regarding the evolution of structures related to speech. Remarkably, the increase in the number of head and neck muscles – and thus of musculoskeletal structures – in human evolution led to a decrease in network density and complexity in humans.

Diversity of the primate musculoskeletal system: modularity, integration, and anatomical variation in humans and their relatives

The concepts of modularity and integration help to explain how morphological transformations during primate evolution enabled modern humans to carry out complex functions such as speaking, chewing, and using tools while also being able to respond to selective pressures. We used anatomical network analysis (AnNA) to assess modularity and integration in the head and upper limb of extant primates. AnNA is the study of complex morphological structures using tools and statistics from network theory. In this context, a module is a group of anatomical elements (e.g., bones and muscles) with more connections (i.e., more integrated) among them than to any other elements outside the group. These modules are identified using community detection algorithms. We analyzed the networks of the head and upper limbs of 20 taxa representing all the major extant primate groups based on dissection of hundreds of specimens, observation of skeletons, and an extensive literature review to ensure that the total sample sizes per taxon would be sufficient to determine the normal muscle phenotype for each taxon. Our results reveal patterns of evolutionary change that are highly relevant for biological and physical anthropology and primatology. For example, the division into left and right orofacial muscle modules seen in humans is not found in bonobos or common chimps, meaning that this division may be related to the more complex, asymmetrical facial expressions displayed by humans.Support or Funding InformationNSF grant 1516557 (2015–16) to Rui Diogo and American Association of Anatomists Innovations Program

Asymmetric facial expressions: revealing richer emotions for embodied conversational agents

ABSTRACTIn this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence–arousal–dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hidden emotions due to social conventions. To achieve such a higher degree of facial expression in a generic way, we propose a new model for mapping the valence–arousal–dominance emotion model onto a set of 12 scalar facial part actions built mostly by combining pairs of antagonist action units from the Facial Action Coding System. The proposed linear model can automatically drive a large number of autonomous virtual humans or support the interactive design of complex facial expressions over time. By design, our approach produces symmetric facial expressions, as expected for most of the emotional spectrum. However, more complex ambivalent feelings can be produced when differing emotions are applied on the left and right sides of the face. We conducted an experiment on static images produced by our approach to compare the expressive power of symmetric and asymmetric facial expressions for a set of eight basic and complex emotions. Results confirm both the pertinence of our general mapping for expressing basic emotions and the significant improvement brought by asymmetry for expressing ambivalent feelings. Copyright © 2013 John Wiley & Sons, Ltd.

Ramsay Hunt Syndrome: Case Report of a Multifaceted Physical Therapy Intervention

Background and purpose: Ramsay Hunt Syndrome is an uncommon diagnosis that has as part of its presentation unilateral facial paralysis. Functional limitations can include: asymmetric facial expression and difficulty speaking, drinking and eating. The most effective treatment is not clear based on published research. The following case study describes the use of various interventions to assist in facial muscle activation in a patient diagnosed with Ramsay Hunt Syndrome. Case description: A 55 year old female patient presented with right sided facial paralysis two months after diagnosis of Ramsay Hunt Syndrome. The patient received progression of range of motion exercises, neuromuscular re-education, soft tissue mobilization, and trials of electrical stimulation as outlined in the current research. In conjunction, she received instruction and application of kinesthetic taping to facilitate and/or inhibit facial muscle activation. The patient was seen for 21 visits over a 14 month time period. Outcomes: House Brackmann grades improved from a grade V to a grade II. Strength increased from 1- to 4+. Functional improvements included improvements in communication, eating/drinking, and facial symmetry at rest. The patient also reported significant improvement in experienced pain. Presence of watery eye, and synkinesis were still mildly observed. Discussion: Limited research is available regarding the treatment of Ramsay Hunt Syndrome. The combination of interventions described in this report was observed to improve muscle activation, facilitate a decrease in pain, and coincide with an improved House Brackmann grade. Further research should include randomized controlled trials to explore the cause and effect of each treatment.

Image Ratio Features for Facial Expression Recognition Application

Video-based facial expression recognition is a challenging problem in computer vision and human-computer interaction. To target this problem, texture features have been extracted and widely used, because they can capture image intensity changes raised by skin deformation. However, existing texture features encounter problems with albedo and lighting variations. To solve both problems, we propose a new texture feature called image ratio features. Compared with previously proposed texture features, e.g., high gradient component features, image ratio features are more robust to albedo and lighting variations. In addition, to further improve facial expression recognition accuracy based on image ratio features, we combine image ratio features with facial animation parameters (FAPs), which describe the geometric motions of facial feature points. The performance evaluation is based on the Carnegie Mellon University Cohn-Kanade database, our own database, and the Japanese Female Facial Expression database. Experimental results show that the proposed image ratio feature is more robust to albedo and lighting variations, and the combination of image ratio features and FAPs outperforms each feature alone. In addition, we study asymmetric facial expressions based on our own facial expression database and demonstrate the superior performance of our combined expression recognition system.

Asymmetrical facial expressions in portraits and hemispheric laterality: A literature review

Studies of facial asymmetry have revealed that the left and the right sides of the face differ in emotional attributes. This paper reviews many of these distinctions to determine how these asymmetries influence portrait paintings. It does so by relating research involving emotional expression to aesthetic pleasantness in portraits. For example, facial expressions are often asymmetrical—the left side of the face is more emotionally expressive and more often connotes negative emotions than the right side. Interestingly, artists tend to expose more of their poser's left cheek than their right. This is significant, in that artists also portray more females than males with their left cheek exposed. Reasons for these psychological findings lead to explanations for the aesthetic leftward bias in portraiture.

The role of structural facial asymmetry in asymmetry of peak facial expressions

Asymmetric facial expression is generally attributed to asymmetry in movement, but structural asymmetry in the face may also affect asymmetry of expression. Asymmetry in posed expressions was measured using image-based approaches in digitised sequences of facial expression in 55 individuals, N=16 men, N=39 women. Structural asymmetry (at neutral expression) was higher in men than women and accounted for .54, .62, and .66 of the variance in asymmetry at peak expression for joy, anger, and disgust expressions, respectively. Movement asymmetry (measured by change in pixel values over time) was found, but was unrelated to peak asymmetry in joy or anger expressions over the whole face and in facial subregions relevant to the expression. Movement asymmetry was negatively related to peak asymmetry in disgust expressions. Sidedness of movement asymmetry (defined as the ratio of summed movement on the left to movement on the right) was consistent across emotions within individuals. Sidedness was found only for joy expressions, which had significantly more movement on the left. The significant role of structural asymmetry in asymmetry of emotion expression and the exploration of facial expression asymmetry have important implications for evolutionary interpretations of facial signalling and facial expressions in general.

High frequency of facial expressions corresponding to confusion, concentration, and worry in an analysis of naturally occurring facial expressions of Americans.

College students were instructed to observe symmetric and asymmetric facial expressions and to report the target's judgment of the "emotion" she or he was expressing, the facial movements involved, and the more expressive side. For both asymmetric and symmetric expressions, some of the most common emotions or states reported are neither included in standard taxonomies of emotion nor studied as important signals. Confusion is the most common descriptor reported for asymmetric expressions and is commonly reported for symmetrical expressions as well. Other frequent descriptors were think-concentrate and worry. Confusion is characterized principally by facial movements around the eyes and has many properties usually attributed to emotions. There was no evidence for lateralization of positive versus negative valenced states.

Personal Experiences and Algorithm of Endoscopically Assisted Subperiosteal Face Lift in Orientals for 5 Years

Orientals are anatomically distinct from Caucasians and are characterized by a thick dermis, a Mongoloid slant of the palpebral fissure, a relatively prominent zygoma and mandible angle, and a relatively flat nose. Given these characteristics, it was believed that the subperiosteal face lift was not suitable for Orientals. However, at our institution, endoscopically assisted subperiosteal face lifts were performed from May of 1994 to October of 1998 on 236 patients; variable pitfalls, as well as satisfying results, were reported. Patient ages ranged from 29 to 66 years (mean age, 55.2 years), and follow-up ranged from 6 to 44 months (mean follow-up, 23 months). All forehead and brow lifts were performed using an endoscopic guide, and routine corrugator resections and procerus myotomies were performed. Three slanted cortical tunnels were made at the corresponding locations on the outer table of the calvarium, and 1-0 nylon or screw suspension and fixation were performed after a 1-cm to 2-cm lift. Midface lifts were performed through lower blepharoplasty incisions and vertical temporal incisions instead of through conventional preauricular and postauricular incisions. Dissections were made subperiosteally and over the deep layers of deep temporal fascia. Malar fat pads were suspended with 1-0 nylon and affixed to deep temporal fascia. Most patients have been satisfied with their postoperative results, but unfavorable results and complications have been reported. Complications were classified as early or late complications or unfavorable results on the basis of the 3-week postoperative evaluation. There were 28 early complications (11.9 percent), consisting of ecchymosis with edema (persisting for up to 4 weeks), paresthesia, lagophthalmos, accentuated Mongoloid slant, small dimpling on the scalp, and scalp fold formation on the fixation site. There were 13 late complications/unfavorable results (5.5 percent), consisting of insufficient lift, exaggeration of sunken upper eyelids, intermittent headaches, itching sensations, and paresthesia on the scalp. The unfavorable results occurred in the patients who had previously undergone blepharoplasty and in those who had a history of foreign body injections into the face, fatty and thick faces, sunken upper eyelids, Mongoloid slants, and asymmetric facial expressions. Through understanding the anatomic characteristics of the Oriental face (i.e., thick dermis, Mongoloid slant of palpebral fissure, prominent zygoma and mandible angle, and flat nose), satisfying results were achieved by appropriate application of the modified procedures. (Plast. Reconstr. Surg. 108: 1768, 2001.)

Neurobiological and cultural aspects of facial asymmetry

Facial asymmetry refers to the fact that the left and the right sides of the face during movement or rest are not identical. This asymmetry can be produced by a range of factors, such as ones that are anatomical, physiological, neurological, psychological, pathological, or sociocultural. Interaction among these various factors is not uncommon and has the potential to confound scientific investigations. When focusing on asymmetrical facial movements and expressions—which is the issue of this article—one has to make sure to control for other types of asymmetries that could confound the observations. To ensure such control, adequate judgment and measurement techniques are necessary. Some of these techniques are briefly described in this article. Furthermore, some explanations for facial asymmetrical movements are considered. From a neuropsychological perspective, it is suggested that asymmetrical facial expressions have some relationship to the functional asymmetry of the brain. But social-psychological and cultural factors, it is argued here, may also play an important role in the phenomenon of asymmetrical facial expressions, although no definitive statements regarding the nature of their role are made at this point.

Asymmetrical Facial Expressions Indeed: A Reply to Dane and Thompson

Asymmetrical Facial Expressions Indeed: A Reply to Dane and Thompson