Local Facial Information Research Articles

CFNet: Facial expression recognition via constraint fusion under multi-task joint learning network

In facial expression recognition (FER), global and local features obtained from the same face may have different recognition accuracy, indicating that they have different advantages in recognition. Existing FER works usually focus on extracting and fusing global and local features to obtain better recognition results. Instead of evaluating the advantages of global and local features before fusion, these methods default to fusing them in the same proportion, which probably leads to mutual suppression of information representation between the two features, and then causes worse recognition ability and scene adaptability. To overcome this weakness, this paper proposes a multi-task joint learning network with a constraint fusion (called CFNet). To leverage the key features extracted from different tasks, CFNet adopts a multi-loss mechanism and a constraint fusion method to automatically assign corresponding weights based on the importance of global and local facial information. Compared with existing models that employ the direct fusion strategy, CFNet has better adaptability for FER in complex scenes. Extensive evaluations show the superior effectiveness of CFNet over state-of-the-art methods on real-world emotion datasets. Specifically, the accuracy scores of CFNet on CK＋, MMI, and RAF-DB datasets are 99.07%, 84.62%, and 87.52%, respectively. The robustness of CFNet is also verified in noisy and blurred scenes.

Dual-Path Deep Fusion Network for Face Image Hallucination.

Along with the performance improvement of deep-learning-based face hallucination methods, various face priors (facial shape, facial landmark heatmaps, or parsing maps) have been used to describe holistic and partial facial features, making the cost of generating super-resolved face images expensive and laborious. To deal with this problem, we present a simple yet effective dual-path deep fusion network (DPDFN) for face image super-resolution (SR) without requiring additional face prior, which learns the global facial shape and local facial components through two individual branches. The proposed DPDFN is composed of three components: a global memory subnetwork (GMN), a local reinforcement subnetwork (LRN), and a fusion and reconstruction module (FRM). In particular, GMN characterize the holistic facial shape by employing recurrent dense residual learning to excavate wide-range context across spatial series. Meanwhile, LRN is committed to learning local facial components, which focuses on the patch-wise mapping relations between low-resolution (LR) and high-resolution (HR) space on local regions rather than the entire image. Furthermore, by aggregating the global and local facial information from the preceding dual-path subnetworks, FRM can generate the corresponding high-quality face image. Experimental results of face hallucination on public face data sets and face recognition on real-world data sets (VGGface and SCFace) show the superiority both on visual effect and objective indicators over the previous state-of-the-art methods.

OAENet: Oriented attention ensemble for accurate facial expression recognition

Facial Expression Recognition (FER) is a challenging yet important research topic owing to its significance with respect to its academic and commercial potentials. In this work, we propose an oriented attention pseudo-siamese network that takes advantage of global and local facial information for high accurate FER. Our network consists of two branches, a maintenance branch that consisted of several convolutional blocks to take advantage of high-level semantic features, and an attention branch that possesses a UNet-like architecture to obtain local highlight information. Specifically, we first input the face image into the maintenance branch. For the attention branch, we calculate the correlation coefficient between a face and its sub-regions. Next, we construct a weighted mask by correlating the facial landmarks and the correlation coefficients. Then, the weighted mask is sent to the attention branch. Finally, the two branches are fused to output the classification results. As such, a direction-dependent attention mechanism is established to remedy the limitation of insufficient utilization of local information. With the help of our attention mechanism, our network not only grabs a global picture but can also concentrate on important local areas. Experiments are carried out on 4 leading facial expression datasets. Our method has achieved a very appealing performance compared to other state-of-the-art methods.

Inducing the use of right eye enhances face-sex categorization performance.

Face recognition ability varies tremendously among neurologically typical individuals. What causes these differences is still largely unknown. Here, we first used a data-driven experimental technique-bubbles-to measure the use of local facial information in 140 neurotypical individuals during a face-sex categorization task. We discovered that the use of the eye and eyebrow area located on the right side of the face image from the observer's viewpoint correlates positively with performance, whereas the use of the left-eye and eyebrow area correlates negatively with performance. We then tested if performance could be altered by inducing participants to use either the right- or the left-eye area. One hundred of these participants thus underwent a 1-hr session of a novel implicit training procedure aimed at inducing the use of specific facial information. Afterward, participants repeated the bubbles face-sex categorization task to assess the changes in use of information and its effect on performance. Participants that underwent right-eye induction used this facial region more than they initially did and, as expected, improved their performance more than the participants who underwent the left-eye induction. This is the first clear evidence of a causal link between the use of specific face information and face recognition ability: Use of right-eye region not only predicts but causes better face-sex categorization. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Real-life experience with personally familiar faces enhances discrimination based on global information.

Despite the agreement that experience with faces leads to more efficient processing, the underlying mechanisms remain largely unknown. Building on empirical evidence from unfamiliar face processing in healthy populations and neuropsychological patients, the present experiment tested the hypothesis that personal familiarity is associated with superior discrimination when identity information is derived based on global, as opposed to local facial information. Diagnosticity and availability of local and global information was manipulated through varied physical similarity and spatial resolution of morph faces created from personally familiar or unfamiliar faces. We found that discrimination of subtle changes between highly similar morph faces was unaffected by familiarity. Contrariwise, relatively more pronounced physical (i.e., identity) differences were more efficiently discriminated for personally familiar faces, indicating more efficient processing of global, as opposed to local facial information through real-life experience.

Recognition of Facial Expressions and Prosodic Cues with Graded Emotional Intensities in Adults with Asperger Syndrome

This study investigated the ability of adults with Asperger syndrome to recognize emotional categories of facial expressions and emotional prosodies with graded emotional intensities. The individuals with Asperger syndrome showed poorer recognition performance for angry and sad expressions from both facial and vocal information. The group difference in facial expression recognition was prominent for stimuli with low or intermediate emotional intensities. In contrast to this, the individuals with Asperger syndrome exhibited lower recognition accuracy than typically-developed controls mainly for emotional prosody with high emotional intensity. In facial expression recognition, Asperger and control groups showed an inversion effect for all categories. The magnitude of this effect was less in the Asperger group for angry and sad expressions, presumably attributable to reduced recruitment of the configural mode of face processing. The individuals with Asperger syndrome outperformed the control participants in recognizing inverted sad expressions, indicating enhanced processing of local facial information representing sad emotion. These results suggest that the adults with Asperger syndrome rely on modality-specific strategies in emotion recognition from facial expression and prosodic information.

Automatic Aspects in Face Perception

We examined the perceptual dependency of local facial information on the whole facial context. In Experiment 1 participants matched a predetermined facial feature that appeared in two sequentially presented faces judging whether it is identical or not, while ignoring an irrelevant dimension in the faces. This irrelevant dimension was either (a) compatible or incompatible with the target's response and (b) same or different in either featural characteristics or metric distance between facial features in the two faces. A compatibility effect was observed for upright but not inverted faces, regardless of the type of change that differentiated between the faces in the irrelevant dimension. Even when the target was presented upright in the inverted faces, to attenuate perceptual load, no compatibility effect was found (Experiment 2). Finally, no compatibility effects were found for either upright or inverted houses (Experiment 3). These findings suggest that holistic face perception is mandatory.

Impaired holistic processing of unfamiliar individual faces in acquired prosopagnosia

Prosopagnosia is an impairment at individualizing faces that classically follows brain damage. Several studies have reported observations supporting an impairment of holistic/configural face processing in acquired prosopagnosia. However, this issue may require more compelling evidence as the cases reported were generally patients suffering from integrative visual agnosia, and the sensitivity of the paradigms used to measure holistic/configural face processing in normal individuals remains unclear. Here we tested a well-characterized case of acquired prosopagnosia (PS) with no object recognition impairment, in five behavioral experiments (whole/part and composite face paradigms with unfamiliar faces). In all experiments, for normal observers we found that processing of a given facial feature was affected by the location and identity of the other features in a whole face configuration. In contrast, the patient's results over these experiments indicate that she encodes local facial information independently of the other features embedded in the whole facial context. These observations and a survey of the literature indicate that abnormal holistic processing of the individual face may be a characteristic hallmark of prosopagnosia following brain damage, perhaps with various degrees of severity.