Facial Information Research Articles

A collaborative filtering method by fusion of facial information features

Personalized recommendation systems fundamentally assess user preferences as a reflection of their emotional responses to items. Traditional recommendation algorithms, focusing primarily on numerical processing, often overlook emotional factors, leading to reduced accuracy and limited application scenarios. This paper introduces a collaborative filtering recommendation method that integrates features of facial information, derived from emotions extracted from such data. Upon user authorization for camera usage, the system captures facial information features. Owing to the diversity in facial information, deep learning methods classify these features, employing the classification results as emotional labels. This approach calculates the similarity between emotional and item labels, reducing the ambiguity inherent in facial information features. The fusion process of facial information takes into account the user’s emotional state prior to item interaction, which might influence the emotions generated during the interaction. Variance is utilized to measure emotional fluctuations, thereby circumventing misjudgments caused by sustained non-interactive emotions. In selecting the nearest neighboring users, the method considers not only the similarity in user ratings but also in their emotional responses. Tests conducted using the Movielens dataset reveal that the proposed method, modeling facial features, more effectively aligns recommendations with user preferences and significantly enhances the algorithm’s performance.

Spatial representation of multidimensional information in emotional faces revealed by fMRI

Face perception is a complex process that involves highly specialized procedures and mechanisms. Investigating into face perception can help us better understand how the brain processes fine-grained, multidimensional information. This research aimed to delve deeply into how different dimensions of facial information are represented in specific brain regions or through inter-regional connections via an implicit face recognition task. To capture the representation of various facial information in the brain, we employed support vector machine decoding, functional connectivity, and model-based representational similarity analysis on fMRI data, resulting in the identification of three crucial findings. Firstly, despite the implicit nature of the task, emotions were still represented in the brain, contrasting with all other facial information. Secondly, the connection between the medial amygdala and the parahippocampal gyrus was found to be essential for the representation of facial emotion in implicit tasks. Thirdly, in implicit tasks, arousal representation occurred in the parahippocampal gyrus, while valence depended on the connection between the primary visual cortex and the parahippocampal gyrus. In conclusion, these findings dissociate the neural mechanisms of emotional valence and arousal, revealing the precise spatial patterns of multidimensional information processing in faces.

Face Recognition and Model Analysis Using CNN Based on Self-Collected Information

Face recognition helps people to determine the features and characteristics of a face, which in turn helps the police to find criminals, reduce the crime rate and ensure the safety of the society. In order to further improve the performance of face recognition, the main objective of this research is to collect a large amount of face information using Convolutional Neural Networks (CNN) to achieve effective recognition. Specifically, first, a code is used in the annotation to define the coordinates of the face. This function converts the classification labels into numbers that can be understood by the model. Second, generate targets in order to parse the annotation file and obtain objects from it. Third, a CNN model is constructed. The features in the data are captured using the adaptive weights of the CNN. In addition, the predictive performance of different models is analysed and compared. In addition, the objective function will transform the face coordinates of the file attached to the boxed list, which helps to acquire the commanded faces in the image file. It is shown through experiments that the proposed model can perform face recognition effectively. The proposed program in this paper can provide valuable thoughts and insights to the field.

Harmonious Mutual Learning for Facial Emotion Recognition

Facial emotion recognition in the wild is an important task in computer vision, but it still remains challenging since the influence of backgrounds, occlusions and illumination variations in facial images, as well as the ambiguity of expressions. This paper proposes a harmonious mutual learning framework for emotion recognition, mainly through utilizing attention mechanisms and probability distributions without utilizing additional information. Specifically, this paper builds an architecture with two emotion recognition networks and makes progressive cooperation and interaction between them. We first integrate self-mutual attention module into the backbone to learn discriminative features against the influence from emotion-irrelevant facial information. In this process, we deploy spatial attention module and convolutional block attention module for the two networks respectively, guiding to enhanced and supplementary learning of attention. Further, in the classification head, we propose to learn the latent ground-truth emotion probability distributions using softmax function with temperature to characterize the expression ambiguity. On this basis, a probability distribution distillation learning module is constructed to perform class semantic interaction using bi-directional KL loss, allowing mutual calibration for the two networks. Experimental results on three public datasets show the superiority of the proposed method compared to state-of-the-art ones.

Rethinking Prior-Guided Face Super-Resolution: A New Paradigm With Facial Component Prior.

Recently, facial priors (e.g., facial parsing maps and facial landmarks) have been widely employed in prior-guided face super-resolution (FSR) because it provides the location of facial components and facial structure information, and helps predict the missing high-frequency (HF) information. However, most existing approaches suffer from two shortcomings: 1) the extracted facial priors are inaccurate since they are extracted from low-resolution (LR) or low-quality super-resolved (SR) face images and 2) they only consider embedding facial priors into the reconstruction process from LR to SR face images, thus failing to explore facial priors to generate LR face image. In this article, we propose a novel pre-prior guided approach that extracts facial prior information from original high-resolution (HR) face images and embeds them into LR ones to obtain HF information-rich LR face images, thereby improving the performance of face reconstruction. Specifically, a novel component hybrid method is proposed, which fuses HR facial components and LR facial background to generate new LR face images (namely, LRmix) via facial parsing maps extracted from HR face images. Furthermore, we design a component hybrid network (CHNet) that learns the LR to LRmix mapping function to ensure that the LRmix can be obtained from LR face images in testing and real-world datasets. Experimental results show that our proposed scheme significantly improves the reconstruction performance for FSR.

Can Perceivers Differentiate Intense Facial Expressions? Eye Movement Patterns.

Recent research on intense real-life faces has shown that although there was an objective difference in facial activities between intense winning faces and losing faces, viewers failed to differentiate the valence of such expressions. In the present study, we explored whether participants could perceive the difference between intense positive facial expressions and intense negative facial expressions in a forced-choice response task using eye-tracking techniques. Behavioral results showed that the recognition accuracy rate for intense facial expressions was significantly above the chance level. For eye-movement patterns, the results indicated that participants gazed more and longer toward the upper facial region (eyes) than the lower region (mouth) for intense losing faces. However, the gaze patterns were reversed for intense winning faces. The eye movement pattern for successful differentiation trials did not differ from failed differentiation trials. These findings provided preliminary evidence that viewers can utilize intense facial expression information and perceive the difference between intense winning faces and intense losing faces produced by tennis players in a forced-choice response task.

Disrupted properties of functional brain networks in major depressive disorder during emotional face recognition: an EEG study via graph theory analysis.

Previous neuroimaging studies have revealed abnormal brain networks in patients with major depressive disorder (MDD) in emotional processing. While any cognitive task consists of a series of stages, little is yet known about the topology of functional brain networks in MDD for these stages during emotional face recognition. To address this problem, electroencephalography (EEG)-based functional brain networks of MDD patients at different stages of facial information processing were investigated in this study. First, EEG signals were collected from 16 patients with MDD and 18 age-, gender-, and education-matched normal subjects when performing an emotional face recognition task. Second, the global field power (GFP) method was employed to divide group-averaged event-related potentials into different stages. Third, using the phase transfer entropy (PTE) approach, the brain networks of MDD patients and normal individuals were constructed for each stage in negative and positive face processing, respectively. Finally, we compared the topological properties of brain networks of each stage between the two groups using graph theory approaches. The results showed that the analyzed three stages of emotional face processing corresponded to specific neurophysiological phases, namely, visual perception, face recognition, and emotional decision-making. It was also demonstrated that depressed patients showed abnormally decreased characteristic path length at the visual perception stage of negative face recognition and normalized characteristic path length in the stage of emotional decision-making during positive face processing compared to healthy subjects. Furthermore, while both the MDD and normal groups' brain networks were found to exhibit small-world network characteristics, the brain network of patients with depression tended to be randomized. Moreover, for patients with MDD, the centro-parietal region may lose its status as a hub in the process of facial expression identification. Together, our findings suggested that altered emotional function in MDD patients might be associated with disruptions in the topological organization of functional brain networks during emotional face recognition, which further deepened our understanding of the emotion processing dysfunction underlying MDD.

Evaluation of AI tools for healthcare networks at the cloud-edge interaction to diagnose autism in educational environments

Physical, social, and routine environments can be challenging for learners with autism spectrum disorder (ASD). ASD is a developmental disorder caused by neurological problems. In schools and educational environments, this disorder may not only hinder a child’s learning, but also lead to more crises and mental convulsions. In order to teach students with ASD, it is essential to understand the impact of their learning environment on their interaction and behavior. Different methods have been used to diagnose ASD in the past, each with their own strengths and weaknesses. Research into ASD diagnostics has largely focused on machine learning algorithms and strategies rather than diagnostic methods. This article discusses many diagnostic techniques used in the ASD literature, such as neuroimaging, speech recordings, facial features, and EEG signals. This has led us to conclude that in schools and educational settings, autism can be diagnosed cheaply, quickly, and accurately through face analysis. To facilitate and speed up the processing of facial information among children in educational settings, we applied the AlexNet architecture designed for edge computing. A fast method for detecting autism spectrum disorders from the face can be applied to educational settings using this structure. While we have investigated a variety of methods, the face can provide us with appropriate information about the disorder. In addition, it can produce more interpretive features. In order to help students in schools who are suffering from this disease, key factors must be considered: potential clinical and therapeutic situations, efficiency, predictability, privacy protection, accuracy, cost-effectiveness, and lack of methodological intervention. The diseases are troublesome, so they should be identified and treated.

Deepfake Creation and Detection of Multimedia Data

As the increasing number of deepfake content poses a growing threat to multimedia integrity, this paper proposes a robust deepfake detection approach based on a hybrid architecture. The proposed framework combines the power of Residual Networks (ResNet) for spatial feature extraction and Long Short-Term Memory (LSTM) with Convolutional Neural Networks (CNN) for temporal dependency modeling. The ResNet component captures complicated patterns in facial and contextual information, whereas the LSTM-CNN module identifies dynamic facial expressions and movements across multiple frames. Transfer learning strategies are used to improve model generalization by combining pre-training on a large dataset with fine-tuning on deepfake data. Experimental evaluations on a variety of deepfake datasets show superior accuracy, precision, and recall, demonstrating the hybrid architecture's efficiency in dealing with the evolving challenges posed by more advanced deepfake generation techniques. In conclusion, our novel deepfake detection framework employs an effective combination of ResNet and LSTM-CNN, demonstrating a promising solution that not only advances the state-of-the-art in multimedia forensics but is also resistant to adversarial attacks. This hybrid model, which effectively combines spatial and temporal information, has the potential to significantly improve the accuracy and reliability of deepfake detection systems in the face of emerging digital threats.

A study on the CNN-based face recognition across various facial angles

Ace recognition is a prominent computer technology with a broad range of applications, such as identity verification and face payment. Over the years, significant advancements have been made in the research and practical implementation of face recognition in academia. However, the variability of face images in real-world scenarios poses challenges to accurate face recognition. Specifically, the variation in facial angles affects the extraction of crucial face information by the model, yet there is a lack of research focusing on the angle factor's impact on face recognition. In this study, we collect and process face datasets, construct a convolutional neural network (CNN) model, capture images containing faces via a camera, detect and track faces from it automatically, and then recognize the detected faces. We investigate the accuracy of face detection and recognition under different angles. Experimental results demonstrate that when the angle ranges from 0 to 30, face detection and recognition achieve excellent accuracy. Within the angle range of 30 to 45, the performance remains within an acceptable range. However, when the angle exceeds 45 and reaches 60, face detection and recognition accuracy decline significantly, resulting in poor performance.

Mimicking Facial Expressions Facilitates Working Memory for Stimuli in Emotion-Congruent Colours.

It is one thing for everyday phrases like "seeing red" to link some emotions with certain colours (e.g., anger with red), but can such links measurably bias information processing? We investigated whether emotional face information (angry/happy/neutral) held in visual working memory (VWM) enhances memory for shapes presented in a conceptually consistent colour (red or green) (Experiment 1). Although emotional information held in VWM appeared not to bias memory for coloured shapes in Experiment 1, exploratory analyses suggested that participants who physically mimicked the face stimuli were better at remembering congruently coloured shapes. Experiment 2 confirmed this finding by asking participants to hold the faces in mind while either mimicking or labelling the emotional expressions of face stimuli. Once again, those who mimicked the expressions were better at remembering shapes with emotion-congruent colours, whereas those who simply labelled them were not. Thus, emotion-colour associations appear powerful enough to guide attention, but-consistent with proposed impacts of "embodied emotion" on cognition-such effects emerged when emotion processing was facilitated through facial mimicry.

The Relative Contribution of Facial and Body Information to the Perception of Cuteness.

Faces and bodies both provide cues to age and cuteness, but little work has explored their interaction in cuteness perception. This study examines the interplay of facial and bodily cues in the perception of cuteness, particularly when these cues convey conflicting age information. Participants rated the cuteness of face-body composites that combined either a child or adult face with an age-congruent or incongruent body alongside manipulations of the head-to-body height ratio (HBR). The findings from two experiments indicated that child-like facial features enhanced the perceived cuteness of adult bodies, while child-like bodily features generally had negative impacts. Furthermore, the results showed that an increased head size significantly boosted the perceived cuteness for child faces more than for adult faces. Lastly, the influence of the HBR was more pronounced when the outline of a body's silhouette was the only available information compared to when detailed facial and bodily features were presented. This study suggests that body proportion information, derived from the body's outline, and facial and bodily features, derived from the interior surface, are integrated to form a unitary representation of a whole person in cuteness perception. Our findings highlight the dominance of facial features over bodily information in cuteness perception, with facial attributes serving as key references for evaluating face-body relationships and body proportions. This research offers significant insights into social cognition and character design, particularly in how people perceive entities with mixed features of different social categories, underlining the importance of congruency in perceptual elements.

Contribution of internal noise and calculation efficiency to face discrimination deficits in older adults

Classification images (CIs) measured in a face discrimination task differ significantly between older and younger observers. These age differences are consistent with the hypothesis that older adults sample diagnostic face information less efficiently, or have higher levels of internal noise, compared to younger adults. The current experiments assessed the relative contributions of efficiency and internal noise to age differences in face discrimination using the external noise masking and double-pass response consistency paradigms. Experiment 1 measured discrimination thresholds for faces embedded in several levels of static white noise, and the resulting threshold-vs.-noise curves were used to estimate calculation efficiency and equivalent input noise: older observers had lower efficiency and higher equivalent input noise than younger observers. Experiment 2 presented observers with two identical sequences of faces embedded in static white noise to measure the association between response accuracy and response consistency and estimate the internal:external (i/e) noise ratio for each observer. We found that i/e noise ratios did not differ significantly between groups. These results suggest that age differences in face discrimination are due to differences in calculation efficiency and additive internal noise, but not to age differences in multiplicative internal noise.

Three-dimensional reconstruction optimization of tunnel face and intelligent extraction of discontinuity orientation based on binocular stereo vision

In the process of grading and dynamically optimizing the design and construction parameters of the surrounding rock mass of a rock tunnel face, efficiently and accurately acquiring the geometrical parameters of the rock discontinuities is an important basic task. To address the problems of time consuming, low accuracy, and high danger associated with traditional methods of obtaining the structural information of rock mass, this paper proposes a method for three-dimensional reconstruction and intelligent information extraction of tunnel face based on binocular stereo vision (BSV). First, the parallel binocular device with a single camera was improved, calibrated using the checkerboard calibration method. By integrating with the semi-global matching algorithm, the BSV based method for the three-dimensional reconstruction of the rock mass of the tunnel face was optimized. Furthermore, based on the results from on-site engineering applications, this study leveraged two parameters, point cloud density and algorithm runtime, to determine the optimal values for the disparity range and window size parameters within the semi-global stereo matching algorithm. This enhancement improved the performance of the 3D reconstruction method based on binocular stereo vision. Finally, efficient and refined intelligent methods for extracting structural parameters of the rock mass were proposed based on k-nearest neighbor search and kernel density estimation. The research results can provide reliable technical support for the intelligent and efficient acquisition of rock mass structural information in rock tunnel engineering faces.

Unconstrained Facial Expression Recognition With No-Reference De-Elements Learning

Most unconstrained facial expression recognition (FER) methods take original facial images as inputs to learn discriminative features by well-designed loss functions, which cannot reflect important visual information in faces. Although existing methods have explored the visual information of constrained facial expressions, there is no explicit modeling of what visual information is important for unconstrained FER. To find out valuable information of unconstrained facial expressions, we pose a new problem of no-reference de-elements learning: we decompose any unconstrained facial image into the facial expression element and a neutral face without the reference of corresponding neutral faces. Importantly, the element provides visualization results to understand important facial expression information and improves the discriminative power of features. Moreover, we propose a simple yet effective <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> e- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> lements <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Net</b> work (DENet) to learn the element and introduce appropriate constraints to overcome no ground truth of corresponding neutral faces during the de-elements learning. We extensively evaluate the proposed method on in-the-wild FER datasets including RAF-DB, AffectNet, SFEW and FERPlus. The comparable results show that our method is promising to improve classification performance and achieves equivalent performance compared with state-of-the-art methods. Also, we demonstrate the strong generalization performance on realistic occlusion and pose variation datasets and the cross-dataset evaluation.

Enhancing Face Recognition with Detachable Self-Supervised Bypass Networks.

Attributed to the development of deep networks and abundant data, automatic face recognition (FR) has quickly reached human-level capacity in the past few years. However, the FR problem is not perfectly solved in case of large poses and uncontrolled occlusions. In this paper, we propose a novel bypass enhanced representation learning (BERL) method to improve face recognition under unconstrained scenarios. The proposed method integrates self-supervised learning and supervised learning together by attaching two auxiliary bypasses, a 3D reconstruction bypass and a blind inpainting bypass, to assist robust feature learning for face recognition. Among them, the 3D reconstruction bypass enforces the face recognition network to encode pose independent 3D facial information, which enhances the robustness to various poses. The blind inpainting bypass enforces the face recognition network to capture more facial context information for face inpainting, which enhances the robustness to occlusions. The whole framework is trained in end-to-end manner with two self-supervised tasks above and the classic supervised face identification task. During inference, the two auxiliary bypasses can be detached from the face recognition network, avoiding any additional computational overhead. Extensive experimental results on various face recognition benchmarks show that, without any cost of extra annotations and computations, our method outperforms state-of-the-art methods. Moreover, the learnt representations can also well generalize to other face-related downstream tasks such as the facial attribute recognition with limited labeled data.

Deep Cross-view Reconstruction GAN based on Correlated Subspace for Multi-view Transformation.

In scenarios where identifying face information in the visible spectrum (VIS) is challenging due to poor lighting conditions, the use of near-infrared (NIR) and thermal (TH) cameras can provide viable alternatives. However, the unique data distribution of images captured by these cameras compared to VIS images presents challenges in matching face identities. To address these challenges, we propose a novel image transformation framework. The framework includes feature extraction from the input image, followed by a transformation network that generates target domain images with perceptual fidelity. Additionally, a reconstruction network preserves original information by reconstructing the original domain image from the extracted features. By considering the correlation between features from both domains, our framework utilizes paired data obtained from the same individual. We apply this framework to two well-established image-to-image transformation models, pix2pix and CycleGAN, known as CRC-pix2pix and CRC-CycleGAN respectively. The versatility of our approach allows extension to other models based on pix2pix or CycleGAN architectures. Our models generate high-quality images while preserving the identity information of the original face. Performance evaluation on TFW and BUAA NIR-VIS datasets demonstrates the superiority of our models in terms of generated image face matching and evaluation metrics such as SSIM, MSE, PSNR, and LPIPS. Moreover, we introduce the CQUPT-VIS-TH dataset, which enriches the paired dataset with thermal-visual face data capturing various angles and expressions.

Facial Prior Guided Micro-Expression Generation.

This paper focuses on the facial micro-expression (FME) generation task, which has potential application in enlarging digital FME datasets, thereby alleviating the lack of training data with labels in existing micro-expression datasets. Despite obvious progress in the image animation task, FME generation remains challenging because existing image animation methods can hardly encode subtle and short-term facial motion information. To this end, we present a facial-prior-guided FME generation framework that takes advantage of facial priors for facial motion generation. Specifically, we first estimate the geometric locations of action units (AUs) with detected facial landmarks. We further calculate an adaptive weighted prior (AWP) map, which alleviates the estimation error of AUs while efficiently capturing subtle facial motion patterns. To achieve smooth and realistic synthesis results, we use our proposed facial prior module to guide motion representation and generation modules in mainstream image animation frameworks. Extensive experiments on three benchmark datasets consistently show that our proposed facial prior module can be adopted in image animation frameworks and significantly improve their performance on micro-expression generation. Moreover, we use the generation technique to enlarge existing datasets, thereby improving the performance of general action recognition backbones on the FME recognition task. Our code is available at https://github.com/sysu19351158/FPB-FOMM.

CTA-Net: A gaze estimation network based on dual feature aggregation and attention cross fusion

Recent work has demonstrated the Transformer model is effective for computer vision tasks. However, the global self-attention mechanism utilized in Transformer models does not adequately consider the local structure and details of images, which may result in the loss of information and local details, causing decreased estimation accuracy in gaze estimation tasks when compared to convolution or sequential stacking methods. To address this issue, we propose a parallel CNNs-Transformer aggregation network (CTA-Net) for gaze estimation, which fully leverages the advantages of the Transformer model in modeling global context while the convolutional neural networks (CNNs) model in retaining local details. Specifically, Transformer and ResNet are deployed to extract facial and eye information, respectively. Additionally, an attention cross fusion (ACFusion) Block is embedded with CNN branch, which decomposes features in space and channels to supplement lost features, suppress noise, and help extract eye features more effectively. Finally, a dual-feature aggregation (DFA) module is proposed to effectively fuse the output features of both branches with the help feature a selection mechanism and a residual structure. Experimental results on the MPIIGaze and Gaze360 datasets demonstrate that our CTA-Net achieves state-of-the-art results.

Automatic Diagnosis of Depression Based on Facial Expression Information and Deep Convolutional Neural Network

Automatic Diagnosis of Depression Based on Facial Expression Information and Deep Convolutional Neural Network