Facial Information Research Articles

Face Adaptation Effects on Non-Configural Face Information.

Inspecting new visual information in a face can affect the perception of subsequently seen faces. In experimental settings for example, previously seen manipulated versions of a face can lead to a clear bias of the participant's perception of subsequent images: Original images are then perceived as manipulated in the opposite direction of the adaptor while images that are more similar to the adaptor are perceived as normal or natural. These so-called face adaptation effects can be a useful tool to provide information about which facial information is processed and stored in facial memory. Most experiments so far used variants of the second-order relationship configural information (e.g., spatial relations between facial features) when investigating these effects. However, non-configural face information (e.g., color) was mainly neglected when focusing on face adaptation, although this type of information plays an important role in face processing. Therefore, we investigated adaptation effects of non-configural face information by employing brightness alterations. Our results provide clear evidence for brightness adaptation effects (Experiment 1). These effects are face-specific to some extent (Experiments 2 and 3) and robust over time (Experiments 4 and 5). They support the assumption that non-configural face information is not only relevant in face perception but also in face retention. Brightness information seems to be stored in memory and thus is even involved in face recognition.

Effects of expression on social perceptions of faces

<p id="p00005">Social perceptions of faces refer to the process in which people perceive others' social characteristics (e.g., personality) based on their face information. Facial expression plays a key role in social perceptions of faces. The local features and structural information are the main factors affecting social perceptions of faces. Facial expression can also impact social perceptions of faces by eliciting the perceivers' emotion and their behavioral tendency. In daily life, facial expressions are very complicated. For example, there might be mixture of multiple expressions, as well as fake expressions; on the other hand, social perceptions of faces are of great subjectivity. Further studies should expand the types of facial expression, and take the perceivers' characteristics into account.

Facial Expression Recognition With Two-Branch Disentangled Generative Adversarial Network

Facial Expression Recognition (FER) is a challenging task in computer vision as features extracted from expressional images are usually entangled with other facial attributes, e.g., poses or appearance variations, which are adverse to FER. To achieve a better FER performance, we propose a model named Two-branch Disentangled Generative Adversarial Network (TDGAN) for discriminative expression representation learning. Different from previous methods, TDGAN learns to disentangle expressional information from other unrelated facial attributes. To this end, we build the framework with two independent branches, which are specific for facial and expressional information processing respectively. Correspondingly, two discriminators are introduced to conduct identity and expression classification. By adversarial learning, TDGAN is able to transfer an expression to a given face. It simultaneously learns a discriminative representation that is disentangled from other facial attributes for each expression image, which is more effective for FER task. In addition, a self-supervised mechanism is proposed to improve representation learning, which enhances the power of disentangling. Quantitative and qualitative results in both in-the-lab and in-the-wild datasets demonstrate that TDGAN is competitive to the state-of-the-art methods.

RESEARCH ON ATHLETES’ PSYCHOLOGICAL REGULATION ABILITY BASED ON BACK PROPAGATION (BP) NEURAL NETWORK ALGORITHM

ABSTRACT Athletes’ psychological control ability directly affects competitions. Therefore, it is necessary to supervise the athletes’ game psychology. Athletes’ game state supervision model is constructed through the facial information extraction algorithm. The homography matrix and the calculation method are introduced. Then, two methods are introduced to solve the rotation matrix from the homography matrix. After the rotation matrix is solved, the method of obtaining the facial rotation angle from the rotation matrix is introduced. The two methods are compared in the simulation data, and the advantages and disadvantages of each algorithm are analyzed to determine the method used in this paper. The experimental results show that the model prediction accuracy reaches 70%, which can effectively supervise the psychological state of athletes. This research study is of great significance to improve the performance of athletes in competitions and improve the application of back propagation (BP) neural network algorithm.

Iris segmentation for non-ideal Iris biometric systems

At present, iris recognition systems are highly demanded for covert applications such as monitoring terrorist activities at public places, walk-through portals, smart cities, etc. In general, these systems use image acquisition setups working under relaxed conditions due to which the quality of acquired images is usually poor. For example, images may contain non-uniform illumination, defocus, blur, reflections and eyelids/eyelashes occlusion. Due to these issues, most contemporary iris segmentation schemes do not perform well. In addition, precise localization of eyes in human face images is also a challenging task. No doubt, wrong localization of eyes may certainly lead to failure of the subsequent system modules. To contribute in this regard, this study offers a robust scheme that functions as follows. First, it supplements the Viola-Jones algorithm with the geometrical information of human face to segment eyes. Next, it preprocesses an eyeimage to enhance its contrast, suppress reflections, smooth down spiky gray-level variations if any and marks a circular region-of-interest (ROI) containing iris. Then, it applies an iterative scheme involving Hough transform to segment iris. Finally, it extracts non-circular iris contours using an effective scheme centered on the Lagrange interpolating polynomial. This scheme has shown improved performance on public face-dataset (CASIA-IrisV4-Distance) and two iris datasets, MMU V1.0 and IITD V1.0. On average, it attained 97.97% accuracy rate on these databases.

Bhavna - Sentiment Analysis using Video Colour Magnification

Facial micro-expressions are visible only for a short amount of time so they are not visible to the naked eye, to make this possible we are combining the advancements in video magnification algorithms, which reveal subtle temporal fluctuations in video frames that are difficult to see with the naked eye, and state of the art facial micro-expression recognition neural network models, to enhance the accuracy of facial emotion detection for sentiment analysis. The objective of this project is to detect any minute changes in the video and detect human expressions accurately. Traditional models are sensitive to the intensity of the facial movements captured, and fail to detect the subtle expressions. In this paper, we propose to increase the precision of microexpression recognition to perceive facial movement information of a video frame. The efficiency of our proposed system is based on the CASME & CASME II dataset.

Effects of posed smiling on memory for happy and sad facial expressions

The perception and storage of facial emotional expressions constitutes an important human skill that is essential for our daily social interactions. While previous research revealed that facial feedback can influence the perception of facial emotional expressions, it is unclear whether facial feedback also plays a role in memory processes of facial emotional expressions. In the present study we investigated the impact of facial feedback on the performance in emotional visual working memory (WM). For this purpose, 37 participants underwent a classical facial feedback manipulation (FFM) (holding a pen with the teeth—inducing a smiling expression vs. holding a pen with the non-dominant hand—as a control condition) while they performed a WM task on varying intensities of happy or sad facial expressions. Results show that the smiling manipulation improved memory performance selectively for happy faces, especially for highly ambiguous facial expressions. Furthermore, we found that in addition to an overall negative bias specifically for happy faces (i.e. happy faces are remembered as more negative than they initially were), FFM induced a positivity bias when memorizing emotional facial information (i.e. faces were remembered as being more positive than they actually were). Finally, our data demonstrate that men were affected more by FFM: during induced smiling men showed a larger positive bias than women did. These data demonstrate that facial feedback not only influences our perception but also systematically alters our memory of facial emotional expressions.

Single trunk multi-scale network for micro-expression recognition

Micro-expressions are the external manifestations of human psychological activities. Therefore, micro-expression recognition has important research and application value in many fields such as public services, criminal investigations, and clinical diagnosis. However, the particular characteristics (e.g., short duration and subtle changes) of micro-expressions bring great challenges to micro-expression recognition. In this paper, we explore the differences in the direction of facial muscle movement when people make different expressions to recognize micro-expressions. We first use optical flow to capture the subtle changes in the facial movement when a micro-expression occurs. Next, we extract facial movement information to an aniso-weighted optical flow image based on anisotropically weighting the horizontal and vertical components of the optical flow. Finally, we feed the aniso-weighted optical flow image into the proposed Single Trunk Multi-scale Network for micro-expression recognition. In particular, the designed multi-scale feature catcher in the network can capture features of micro-expressions with different intensities. We conduct extensive experiments on four spontaneous micro-expression datasets, and the experiment results show that our proposed method is competitive and effective.

Effects of an Audiovisual Emotion Perception Training for Schizophrenia: A Preliminary Study

Individuals with schizophrenia show a reduced ability to integrate facial and vocal information in emotion perception. Although emotion perception has been a target for treatment, no study has yet examined the effect of multimodal training on emotion perception in schizophrenia. In the present study, we developed an audiovisual emotion perception training and test in which a voice and a face were simultaneously presented, and subjects were asked to judge whether the emotions of the voice and the face matched. The voices were either angry or happy, and the faces were morphed on a continuum ranging from angry to happy. Sixteen patients with schizophrenia participated in six training sessions and three test sessions (i.e., pre-training, post-training, and generalization). Eighteen healthy controls participated only in pre-training test session. Prior to training, the patients with schizophrenia performed significantly worse than did the controls in the recognition of anger; however, following the training, the patients showed a significant improvement in recognizing anger, which was maintained and generalized to a new set of stimuli. The patients also improved the recognition of happiness following the training, but this effect was not maintained or generalized. These results provide preliminary evidence that a multimodal, audiovisual training may yield improvements in anger perception for patients with schizophrenia.

A spatio-temporal integrated model based on local and global features for video expression recognition

Facial expressions can be represented largely by the dynamic variations of important facial expression parts, i.e., eyebrows, eyes, nose, and mouth. The features of these parts are regarded as local features. However, facial global information is also useful for recognition because it is a necessary complement to local features. In this paper, a spatio-temporal integrated model that jointly learns local and global features is proposed for video expression recognition. Firstly, to capture the action of facial key units, a spatio-temporal attention part-gradient-based hierarchical bidirectional recurrent neural network (spatio-temporal attention PGHRNN) is constructed. It can capture the dynamic variations of gradients around facial landmark points. In addition, a new kind of spatial attention mechanism is introduced to recalibrate the features of facial various parts adaptively. Secondly, to complement the local features extracted by the spatio-temporal attention PGHRNN, a squeeze-and-excitation residual network of 50 layers with long short-term memory network (SE-ResNet-50-LSTM) is used as a global feature extractor and classifier. Finally, to integrate the local and global features and improve the performance of facial expression recognition, a joint adaptive fine-tuning method (JAFTM) is proposed to combine the two networks, which can adaptively adjust the network weights. Extensive experiments demonstrate that our proposed model can achieve a superior recognition accuracy of 98.95% on CK + for 7-class facial expressions and 85.40% on MMI database, which outperforms other state-of-the-art methods.

Culture shapes emotion perception from faces and voices: changes over development

ABSTRACT The perception of multisensory emotion cues is affected by culture. For example, East Asians rely more on vocal, as compared to facial, affective cues compared to Westerners. However, it is unknown whether these cultural differences exist in childhood, and if not, which processing style is exhibited in children. The present study tested East Asian and Western children, as well as adults from both cultural backgrounds, to probe cross-cultural similarities and differences at different ages, and to establish the weighting of each modality at different ages. Participants were simultaneously shown a face and a voice expressing either congruent or incongruent emotions, and were asked to judge whether the person was happy or angry. Replicating previous research, East Asian adults relied more on vocal cues than did Western adults. Young children from both cultural groups, however, behaved like Western adults, relying primarily on visual information. The proportion of responses based on vocal cues increased with age in East Asian, but not Western, participants. These results suggest that culture is an important factor in developmental changes in the perception of facial and vocal affective information.

Processing of Task-Irrelevant Race Information is Associated with Diminished Cognitive Control in Black and White Individuals

The race of an individual is a salient physical feature that is rapidly processed by the brain and can bias our perceptions of others. How the race of others explicitly impacts our actions toward them during intergroup contexts is not well understood. In the current study, we examined how task-irrelevant race information influences cognitive control in a go/no-go task in a community sample of Black (n = 54) and White (n = 51) participants. We examined the neural correlates of behavioral effects using functional magnetic resonance imaging and explored the influence of implicit racial attitudes on brain-behavior associations. Both Black and White participants showed more cognitive control failures, as indexed by dprime, to Black versus White faces, despite the irrelevance of race to the task demands. This behavioral pattern was paralleled by greater activity to Black faces in the fusiform face area, implicated in processing face and in-group information, and lateral orbitofrontal cortex, associated with resolving stimulus-response conflict. Exploratory brain-behavior associations suggest different patterns in Black and White individuals. Black participants exhibited a negative association between fusiform activity and response time during impulsive errors to Black faces, whereas White participants showed a positive association between lateral OFC activity and cognitive control performance to Black faces when accounting for implicit racial associations. Together our findings propose that attention to race information is associated with diminished cognitive control that may be driven by different mechanisms for Black and White individuals.

Product Proliferation under Rational Inattention: Application to Health Insurance

In markets with complicated products such as insurance, why do firms offer many products even when consumers appear to receive little benefit? We show that when consumers face information acquisition costs, firms may have an incentive to introduce many undifferentiated products. This allows firms to gain market share and increase markups. We document initial evidence consistent with the model using data from Medicare prescription drug insurance. Insurers that offer more duplicate or similar plans have higher-cost plans. These results suggest a role for policymakers to restrict product proliferation in markets with complicated products.

Effects of Facial and Vocal information on Online Personality Assessment: Their Order and Relative Effects, and Gender Differences Between Targets

본 연구는 평가대상자의 성격에 대한 평가에서 얼굴과 목소리의 제시순서의 효과, 이 두 정보의 상대적 영향력, 평가대상자의 성별이 미치는 효과를 알아보고자 했다. 연구 1에서는 104명(남: 14명, 여: 90명)의 대학생이 남녀 각각 6명 총 12명의 평가대상자의 얼굴과 목소리 정보를 서로다른 순서로 제시받고 그들의 성격을 성격 Big Five 척도에서 평가했다. 그 결과, 외향성, 우호성, 성실성에서 초두효과가 나타나, 전반적으로 최신효과보다는 초두효과가 인상 형성에 더 강력하다는 것을 보여주었다. 또한 외향성 평가에서는 얼굴이 목소리보다 더 큰 영향을 미쳤다. 우호성과성실성 평가에 정보의 순서와 종류가 미치는 효과는 평가대상자의 성별에 따라 달랐다. 연구 2에서는 123명(남: 34명, 여: 89명)의 대학생이 내향적이거나 외향적인 남녀 각 2명 총 4명의 얼굴과목소리 정보를 받고 그들의 외향성 정도를 평가하였다. 그 결과, 연구 1처럼 외향성 평가에서얼굴의 효과가 목소리의 효과가 더 컸으며 특히, 남성보다 여성의 평가에서 그 효과가 더 컸다. 이러한 결과가 갖는 의미를 기존 연구들과 비교해서 논의하였다.

Multimodal Low Resolution Face and Frontal Gait Recognition from Surveillance Video

Biometric identification using surveillance video has attracted the attention of many researchers as it can be applicable not only for robust identification but also personalized activity monitoring. In this paper, we present a novel multimodal recognition system that extracts frontal gait and low-resolution face images from frontal walking surveillance video clips to perform efficient biometric recognition. The proposed study addresses two important issues in surveillance video that did not receive appropriate attention in the past. First, it consolidates the model-free and model-based gait feature extraction approaches to perform robust gait recognition only using the frontal view. Second, it uses a low-resolution face recognition approach which can be trained and tested using low-resolution face information. This eliminates the need for obtaining high-resolution face images to create the gallery, which is required in the majority of low-resolution face recognition techniques. Moreover, the classification accuracy on high-resolution face images is considerably higher. Previous studies on frontal gait recognition incorporate assumptions to approximate the average gait cycle. However, we quantify the gait cycle precisely for each subject using only the frontal gait information. The approaches available in the literature use the high resolution images obtained in a controlled environment to train the recognition system. However, in our proposed system we train the recognition algorithm using the low-resolution face images captured in the unconstrained environment. The proposed system has two components, one is responsible for performing frontal gait recognition and one is responsible for low-resolution face recognition. Later, score level fusion is performed to fuse the results of the frontal gait recognition and the low-resolution face recognition. Experiments conducted on the Face and Ocular Challenge Series (FOCS) dataset resulted in a 93.5% Rank-1 for frontal gait recognition and 82.92% Rank-1 for low-resolution face recognition, respectively. The score level multimodal fusion resulted in 95.9% Rank-1 recognition, which demonstrates the superiority and robustness of the proposed approach.

Small data assisting face image illumination normalization

Image transfer based on deep learning methods can achieve good results in face illumination processing. However, data constraints and generalization ability restrict the further development of these methods in this field. In this paper, we propose small data assisting face image illumination normalization. For data constraints, we train the network model on a small number of image pairs. In terms of generalization ability, the proposed normalization network parameters are different for processing different face images. Small data learning can provide prior knowledge, and the reconstruction process can guide detail generation. Therefore, the small data learning network and the reconstruction network are complementary to each other in image generating mode when only a small quantity of data is available. We use this network mode to normalize the illumination and reconstruct the super-resolution of face images. After illumination normalization, super-resolution reconstruction can obtain more precise face information and further improve the face recognition rate. Experiments show that the proposed method has good normalization performance when only 500 face image pairs are used for training.

Resistance to facial recognition payment in China: The influence of privacy-related factors

As mobile payment technology is at a nascent stage, the use of facial recognition payment (FRP) services is gradually penetrating the lives of Chinese people. Although the FRP system may have advantages over other payment technologies, a civil lawsuit over refusing to submit facial information and a series of illegal activities related to selling facial information have raised the public's privacy concerns, which might further engender Chinese users' resistance towards FRP. Based on privacy calculus theory and innovation resistance theory, this study builds a research model of FRP and examines it by using a cross-sectional study with 1200 Chinese users. The findings demonstrate that the perceived effectiveness of privacy policy has significant relationships with privacy control, perceived privacy risk, perceived benefits, and resistance. Both privacy control and perceived privacy risk are significantly related to privacy concerns. There is also a significant relationship between the perceived privacy risk and resistance to FRP. Meanwhile, privacy concerns positively affect user resistance, while perceived benefits negatively affect user resistance. In contrast to previous research, the perceived privacy risk has a positive impact on the perceived benefits. This study offers cutting-edge contributions to both academia and industry.

Assessing the effect of facial disguises on forensic facial comparison by morphological analysis.

Disguises are commonly used to mask a person's facial appearance in areas under closed-circuit television (CCTV) surveillance. While many studies attempted to understand the effects of disguises, such as hats and glasses, on facial recognition, limited studies have looked at disguises in forensic facial comparison. The aim of this study was to compare the outcomes of forensic facial comparison by morphological analysis (MA) in a CCTV sample with sunglasses and brimmed caps. The sample was obtained from the Wits Face Database and organized into 81 face pools of one target facial image wearing a disguise (cap or sunglasses) and 10 potential matching images. MA was conducted across face pools, and confusion matrices were used to assess the outcomes. Surprisingly, sunglasses had limited effect on MA performance both in accuracy (90.4%) and in reliability (κ=0.798), while caps markedly decreased both accuracy (68.1%) and reliability (κ=0.639). Error rates were associated primarily with false negatives in both samples (caps: 42.4%; sunglasses: 16.1%) despite the sample distribution favoring false-positive errors, which were very low (caps: 0.6%; sunglasses: 0%). Similarly to other studies, hats and caps were more harmful to correct identification when compared to sunglasses, which actually resulted in better accuracy than regular CCTV recordings. The effect of brimmed caps on accuracy was attributed to the overall loss of facial information caused. On training analysts, it may be helpful to instruct purposefully avoiding overreliance on easily disguised facial features, as other regions of the face also contain substantial feature information.

Local and Global Perception Generative Adversarial Network for Facial Expression Synthesis

Facial expression synthesis has gained increasing attention with the development of Generative Adversarial Networks (GANs). However, it is still very challenging to generate high-quality facial expressions since the overlapping and blur commonly appear in the generated facial images especially in the regions with rich facial features such as eye and mouth. Generally, existing methods mainly consider the face as a whole in facial expression synthesis without paying specific attention to the characteristics of facial expressions. In fact, according to the physiological and psychological research, the differences of facial expressions often appear in crucial regions such as eye and mouth. Motivated by this observation, a novel end-to-end facial expression synthesis method called Local and Global Perception Generative Adversarial Network (LGP-GAN) with a two-stage cascaded structure is proposed in this paper which is designed to extract and synthesize the details of the crucial facial regions. LGP-GAN can combine the generated results from the global network and local network into the corresponding facial expressions. In Stage I, LGP-GAN utilizes local networks to capture the local texture details of the crucial facial regions and generate local facial regions, which fully explores crucial facial region domain information in facial expressions. And then LGP-GAN uses a global network to learn the whole facial information in Stage II to generate the generate final facial expressions building upon local generated results from Stage I. We conduct qualitative and quantitative experiments on the commonly used public database to verify the effectiveness of the proposed method. Experimental results show the superiority of the proposed method over the state-of-the-art methods.

Within-person variability can improve the identification of unfamiliar faces across changes in viewpoint.

Unfamiliar face identification is concerningly error prone, especially across changes in viewing conditions. Within-person variability has been shown to improve matching performance for unfamiliar faces, but this has only been demonstrated using images of a front view. In this study, we test whether the advantage of within-person variability from front views extends to matching to target images of a face rotated in view. Participants completed either a simultaneous matching task (Experiment 1) or a sequential matching task (Experiment 2) in which they were tested on their ability to match the identity of a face shown in an array of either one or three ambient front-view images, with a target image shown in front, three-quarter, or profile view. While the effect was stronger in Experiment 2, we found a consistent pattern in match trials across both experiments in that there was a multiple image matching benefit for front, three-quarter, and profile-view targets. We found multiple image effects for match trials only, indicating that providing observers with multiple ambient images confers an advantage for recognising different images of the same identity but not for discriminating between images of different identities. Signal detection measures also indicate a multiple image advantage despite a more liberal response bias for multiple image trials. Our results show that within-person variability information for unfamiliar faces can be generalised across views and can provide insights into the initial processes involved in the representation of familiar faces.