Facial Feature Extraction Research Articles

A Student Facial Expression Recognition Model Based on Multi-Scale and Deep Fine-Grained Feature Attention Enhancement.

In smart classroom environments, accurately recognizing students' facial expressions is crucial for teachers to efficiently assess students' learning states, timely adjust teaching strategies, and enhance teaching quality and effectiveness. In this paper, we propose a student facial expression recognition model based on multi-scale and deep fine-grained feature attention enhancement (SFER-MDFAE) to address the issues of inaccurate facial feature extraction and poor robustness of facial expression recognition in smart classroom scenarios. Firstly, we construct a novel multi-scale dual-pooling feature aggregation module to capture and fuse facial information at different scales, thereby obtaining a comprehensive representation of key facial features; secondly, we design a key region-oriented attention mechanism to focus more on the nuances of facial expressions, further enhancing the representation of multi-scale deep fine-grained feature; finally, the fusion of multi-scale and deep fine-grained attention-enhanced features is used to obtain richer and more accurate facial key information and realize accurate facial expression recognition. The experimental results demonstrate that the proposed SFER-MDFAE outperforms the existing state-of-the-art methods, achieving an accuracy of 76.18% on FER2013, 92.75% on FERPlus, 92.93% on RAF-DB, 67.86% on AffectNet, and 93.74% on the real smart classroom facial expression dataset (SCFED). These results validate the effectiveness of the proposed method.

A Review of Traditional Methods and Deep Learning for Face Recognition

Face recognition technology is a key research direction in artificial intelligence and computer vision. It first originated in the 1960s, when Bledsoe et al. proposed and studied computer programming based on face recognition. It is divided into three steps: face detection, feature extraction, and face verification. This paper briefly introduces the wide application of face recognition in various fields such as vulnerable groups, analyses its characteristics and progress from the viewpoints of traditional learning and deep learning, and focuses on the methods based on face feature extraction (including geometric features, texture features, local features, etc.). From the previous studies appear some problems connected to environmental changes, such as light change, facial occlusion, posture and privacy leakage, etc. This paper will give the possible reasons for these problems and give the possible development direction of the future by exploring the current face recognition technology to provide a reference for the related research of face recognition technology.

An Enhanced LBPH Algorithm for Robust Face Feature Extraction

From security systems to human-computer interaction, face recognition technology is a key component in many different applications. In this domain, the local binary pattern histogram (lbph) algorithm has shown great promise due to its ability in texture-based feature extraction and robustness. In this study, the performance of lbph algorithm is improved by optimising important parameters: radius, number of neighbours and grid configuration. Tweaking these parameters systematically enable us to get the most success out of this algorithm. Lbph algorithm parameters has been tuned into a 5x7 dimensional matrix, which gives us total of 35 grids with equal width and height pixels. In other words, one central pixel plus 34 neighbouring pixels where the radii of 3 square neighbourhoods can be adjusted. The study combines the experimental exploration with fine-tuning via machine learning approaches to optimize lbph algorithm. Finally, we have provided experimental results on intra-class and inter-class feature distribution analysis conducted on selected images taken under constraints and unconstraint environments. The performance of our novel 34N-LBPH algorithm showed very low values by obtaining intra-class average means of 0.031, 0.078, and 0.101 for three classes under constraint environment indicating the proposed 34n-lbph is robust for facial feature extraction. The findings suggest that our enhanced algorithm, 34 neighbour linear binary pattern Histogram (34N-LBPH) can effectively handle variations in lighting, expressions and occlusions, contributing to the advancement of facial feature extraction for face recognition.

Application of LSTM-based Body Fatigue Detection Algorithm in Tai Chi Training

Tai Chi training sessions are often lengthy, and athletes are prone to experiencing fatigue during the process. Timely detection of body fatigue can help athletes prevent injuries caused by excessive fatigue. This study combines Long Short-Term Memory (LSTM) networks with facial muscle activity detection models to propose a novel fatigue detection algorithm. In this algorithm, the limitations of LSTM networks in capturing future information are addressed by introducing an improved LSTM network model and combining attention mechanisms to highlight the important features of physical fatigue. This study utilizes hyperspectral imaging technology to extract real-time muscle fatigue signals from the faces of subjects in the dataset. Performance validation of the proposed model shows that it effectively extracts facial fatigue features with a detection accuracy of 97.67% and a recall rate of 96.78%, outperforming other existing models in this field. The model constructed through research has excellent performance and has broad application prospects in current and future technological development due to its high flexibility and adaptability, providing support and innovation momentum for different industries.

A Hybrid Approach for Face Morphing Detection

Background: In biometrics, one of the most popular study topics is the detection of face morphing attacks. However, because present methods are unable to capture significant feature changes, they are unable to strike the correct balance between accuracy and complexity. Survey investigation and analysis have shown that the existing method of face morphing detection take a bit longer time to detect the image attack due to the high computation required by facial feature extraction approaches. Conversely, further study is needed to develop a model to enhance the computational time and accuracy of the current face morphing recognition methods. The paper developed a hybrid model for face morphing detection. The FERET database was created to aid in the evaluation and development of algorithms. Local Binary Pattern (LBP) was used as feature extraction algorithm and Residue Number System (RNS) was introduced to reduce the lengthy computational time of LBP during the extraction of images. The classification accuracy of 98% was achieved for the FERET database, while an accuracy of 96% was achieved for the FRGCv2 database. An average training time of 0.0532seconds was recorded for the FERET database, while an average training time of 0.0582seconds was achieved for the FRGCv2 database. The study concluded that the high dimensionality of LBP was well reduced and optimized by the RNS algorithm, which improved the performance of face morphing recognition.

Robust Face Recognition System Based on Deep Facial Feature Extraction and Machine Learning

Facial Recognition is a common challenge in research for security applications. Wide-ranging applications of this technology include biometrics, security data, access to controlled locations, smart cards, and surveillance systems. A practical approach to identifying individuals depends on factors such as partial face occlusion, illumination, age, expression, makeup, and poses. These complex face recognition variables affect most face recognition systems. In this paper, we propose a deep transfer learning for facial features based on the pose and illumination variations of celebrities’ faces in a free environment for face recognition. The new approach that combines deep learning methods for feature extraction with machine learning classification methods. Also, studies the performance of the pre-trained model (VGGFace2 with ResNet-50 model weights) for feature extraction with a Multilayer Perceptron Classifier (MLP), Decision Tree, and Bootstrap Aggregation (Bagging) to perform classification. The convolution neural network has lately achieved excellent advancement in facial recognition. The outcome indicated that VGGFace2 with MLP obtained more precision and provided the best results (Accuracy, F-measure, Recall, and Precision). The results for all metric for the proposed model is 99%. The benchmark for this model is a dataset of 105 people’s faces.

Deep Nonnegative Matrix Factorization With Beta Divergences.

Deep nonnegative matrix factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all existing deep NMF models and algorithms have primarily centered their evaluation on the least squares error, which may not be the most appropriate metric for assessing the quality of approximations on diverse data sets. For instance, when dealing with data types such as audio signals and documents, it is widely acknowledged that ß-divergences offer a more suitable alternative. In this article, we develop new models and algorithms for deep NMF using some ß-divergences, with a focus on the Kullback-Leibler divergence. Subsequently, we apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.

Improved Histogram of Oriented Gradient (HOG) Feature Extraction for Facial Expressions Classification

Facial expression classification system is one of the implementations of machine learning (ML) that takes facial expression datasets, undergoes training, and then utilizes the trained results to recognize facial expressions in new facial images. The recognized facial expressions include anger, contempt, disgust, fear, happy, sadness, and surprise expressions. The method employed for facial feature extraction utilizes histogram-oriented gradient (HOG). This study proposes an enhancement method for HOG feature extraction by reducing the feature dimension into multiple sub-features based on gradient orientation intervals, referred to as HOG channel (HOG-C). Classifier testing techniques are divided into two methods for comparison—support vector machines (SVM) with HOG features and SVM with HOG-C features. The testing results demonstrate that SVM with HOG achieves an accuracy of 99.9% with an average training time of 18.03 minutes, while SVM with HOG-C attains a 100% accuracy with an average training time of 18.09 minutes. The testing outcomes reveal that the implementation of SVM with HOG-C successfully enhances accuracy for facial expression classification.

Combating deepfakes: a comprehensive multilayer deepfake video detection framework

AbstractDeepfakes represent a class of synthetic media crafted with the aid of advanced deep learning techniques that exhibit an unparalleled degree of authenticity. The rapid advancement in Artificial Intelligence (AI) has contributed to an increase in the prevalence of deepfakes on the internet, consequently amplifying the spread of misinformation among the public. Consequently, the detection of deepfakes has become a pressing concern. In this context, we put forth a comprehensive framework for deepfake video detection, which is built upon three distinct layers. The first layer, termed as the RGB features extraction layer, is designed to identify potential signs of forgery within the spatial domain of analogous video frames. The second layer, known as the GAN features extraction layer, focuses on the extraction of forgery fingerprints in the high-frequency region. This layer is specifically engineered to detect the fingerprints left by the Generative Adversarial Network (GAN) process in fake videos and the traces of the imaging process in genuine videos. The third and final layer, referred to as the facial region intra-frame inconsistency feature extraction layer, is dedicated to uncovering the anomalies associated with the manipulation process. This is achieved by extracting features from both the inner and outer regions of the manipulated portion of a frame. The extensive experimental evaluations have underscored the superior performance of proposed approach in comparison to existing state-of-the-art methods.

PointSurFace: Discriminative point cloud surface feature extraction for 3D face recognition

Due to the geometric information in the 3D face data, the 3D face recognition methods exhibit better robustness against the physical attacks compared to the 2D recognition methods. In this paper, we propose PointSurFace, a 3D face point cloud recognition method integrated with a specially designed facial surface feature extraction (SurF) module. In PointSurFace, SurF comprehensively combines the coordinate positions, normal vectors, and angle relationships, etc., of each patch in the 3D face point cloud, to explore the explicit local geometric information of 3D faces and generate facial surface features. Subsequently, the surface features are fed into an encoder consisting of four set abstraction (SA) modules and three Inverted Residual MLP (InvResMLP) modules to extract more discriminative 3D facial features, where InvResMLP could suppress the overfitting and reduce information loss in the model training. In addition, PointSurFace utilizes channel fusion operations to integrate position information into the extracted 3D facial features, further enhancing the model performance. Experimental results in 3D facial recognition and verification across multiple datasets demonstrate that PointSurFace achieves the best 3D face recognition performance to date. For example, on Lock3DFace, it achieves the state-of-the-art recognition and verification accuracy of 90.03% and 80.01%, respectively, surpassing the previous methods by 2.02% and 3.19%. The ablation studies evaluate each module of PointSurFace, suggesting that the proposed surface feature extraction module significantly enhances the model performance.

Refining CycleGAN with attention mechanisms and age-Aware training for realistic Deepfakes

In the evolving landscape of deep learning technologies, the emergence of Deepfakes and synthetic media is becoming increasingly prominent within digital media production. This research addresses the limitations inherent in existing face image generation algorithms based on Generative Adversarial Networks (GAN), particularly the challenges of domain irrelevancy and inadequate facial detail representation. The study introduces an enhanced face image generation algorithm, aiming to refine the CycleGAN framework. The enhancement involves a two-fold strategy: firstly, the generator's architecture is refined through the integration of an attention mechanism and adaptive residual blocks, enabling the extraction of more nuanced facial features. Secondly, the discriminator's accuracy in distinguishing real from synthetic images is improved by incorporating a relative loss concept into the loss function. Additionally, this study presents a novel model training approach that incorporates age constraints, thereby mitigating the effects of age variations on the synthesized images. The effectiveness of the proposed algorithm is empirically validated through comparative analysis with existing methodologies, utilizing the CelebA dataset. The results demonstrate that the proposed algorithm significantly enhances the realism of generated face images, outperforming current methods in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), while also achieving notable improvements in subjective visual quality. The implementation of this advanced method is anticipated to substantially elevate the efficiency and quality of digital media production, contributing positively to the broader field of digital media creation.

Advanced Techniques for Real-time Facial Expression Recognition Using Deep Learning

Abstract—Developing systems that can automatically recognize and interpret human emotions from facial expressions is the aim of the quickly expanding field of facial emotion identification and detection research. This technology finds applications in a wide range of areas, including as healthcare, marketing, security, and human-computer interface. Using computer vision and machine learning algorithms, facial emotion recognition systems analyze a face’s features and classify it into numerous emotional categories, including joyful, sorrowful, angry, fearful, and surprised.The three steps in the multi-step process that goes into identifying facial emotions are face detection, facial feature extraction, and emotion categorization. Thanks to recent advances in deep learn- ing, facial emotion detection systems can now identify emotions with high resilience and precision. Further, the development of real-time face expression recognition systems has opened up new avenues for applications such as sentiment analysis, emotional intelligence, and affective computing. This technology could fundamentally alter human-machine interactions and open the way to more compassionate and personalized relationships.A multitude of applications, including virtual assistants, mental health aids, and human-centered technology, will be greatly impacted by the development of systems for identifying and de- tecting facial expressions. Artificial intelligence (AI) technologies that recognize emotions allow people to interact with the digital world in more intelligent and flexible ways. But the complexity of emotion identification lies in the fact that it requires context and geometric elements in addition to facial expressions.

CAGNet: a context-aware graph neural network for detecting social relationships in videos

Social relationships, such as parent-offspring and friends, are crucial and stable connections between individuals, especially at the person level, and are essential for accurately describing the semantics of videos. In this paper, we analogize such a task to scene graph generation, which we call video social relationship graph generation (VSRGG). It involves generating a social relationship graph for each video based on person-level relationships. We propose a context-aware graph neural network (CAGNet) for VSRGG, which effectively generates social relationship graphs through message passing, capturing the context of the video. Specifically, CAGNet detects persons in the video, generates an initial graph via relationship proposal, and extracts facial and body features to describe the detected individuals, as well as temporal features to describe their interactions. Then, CAGNet predicts pairwise relationships between individuals using graph message passing. Additionally, we construct a new dataset, VidSoR, to evaluate VSRGG, which contains 72 h of video with 6276 person instances and 5313 relationship instances of eight relationship types. Extensive experiments show that CAGNet can make accurate predictions with a comparatively high mean recall (mRecall) when using only visual features.

Genetic algorithms and deep learning for unique facial landmark-based key generation

Generating secure secret keys remains essential in the realm of biometric authentication systems. Traditional methods have often suffered from inefficiency, insecurity, or the requirement for additional hardware. In this study, an innovative approach to secret key generation is proposed, leveraging deep learning, facial feature extraction, genetic algorithms, and linear feedback shift registers (LFSRs). These techniques are combined to create robust, unique keys based on users' facial features. A convolutional neural network (CNN) is employed for the extraction of facial features from user images and for the optimization of LFSR parameters using a genetic algorithm. Furthermore, another neural network is utilized to establish a connection between facial features and the LFSR-generated output, resulting in the secret key. Rigorous evaluation of the method is conducted across various facial datasets, with a comparison against existing approaches. The results demonstrate the effectiveness of the method, yielding secret key length of 92,706 characterized by an entropy value of 3.24, a low average correlation value of 0.1554, and a high level of security. This research represents a significant advancement in secure biometric authentication, addressing the limitations of conventional key generation methods.

Face Recognition Using LBPH and CNN

Objective:: The purpose of this paper was to use Machine Learning (ML) techniques to extract facial features from images. Accurate face detection and recognition has long been a problem in computer vision. According to a recent study, Local Binary Pattern (LBP) is a superior facial descriptor for face recognition. A person's face may make their identity, feelings, and ideas more obvious. In the modern world, everyone wants to feel secure from unauthorized authentication. Face detection and recognition help increase security; however, the most difficult challenge is to accurately recognise faces without creating any false identities. Methods:: The proposed method uses a Local Binary Pattern Histogram (LBPH) and Convolution Neural Network (CNN) to preprocess face images with equalized histograms. Results:: LBPH in the proposed technique is used to extract and join the histogram values into a single vector. The technique has been found to result in a reduction in training loss and an increase in validation accuracy of over 96.5%. Prior algorithms have been reported with lower accuracy when compared to LBPH using CNN. Conclusion:: This study demonstrates how studying characteristics produces more precise results, as the number of epochs increases. By comparing facial similarities, the vector has generated the best result.

Wheelchair Navigation Enhanced by Machine Learning Eye-Gaze Control

Abstract: This study focuses on the development of an eye-controlled wheelchair system tailored for individuals with motor disabilities, with the overarching goal of augmenting their mobility and autonomy. A multifaceted approach was adopted, incorporating meticulous data collection and preprocessing techniques, notably encompassing real-time face detection and feature extraction. Various machine learning algorithms, including KNN, Decision Trees, Random Forests, and SVM, were meticulously trained to anticipate user commands. Through rigorous performance evaluation, the system demonstrated remarkable accuracy and precision, with KNN and SVM emerging as the top-performing models. Notably, this innovative system exhibits significant potential in enhancing accessibility and fostering independence, thereby exemplifying notable strides in the realm of assistive technology. The integration of advanced data processing methodologies, machine learning techniques, and real-time prediction mechanisms facilitates a seamless and intuitive navigation experience, thus holding promise for substantially enriching the quality of life for individuals grappling with mobility disabilities. This research underscores the pivotal role of interdisciplinary collaboration and technological innovation in devising solutions that cater to the diverse needs of individuals with disabilities, thereby advocating for inclusivity and empowerment in society.

VisageVision: Next-Generation Facial Feature Extraction Tool

VisageVision: Next-Generation Facial Feature Extraction Tool

Gender Classification Using Viola Jones, Orthogonal Difference Local Binary Pattern and Principal Component Analysis

Facial recognition is currently a widely discussed topic, particularly in the context of gender classification. Facial recognition by computers is more complex and time-consuming compared to humans. There is ongoing research on facial feature extraction for gender classification. Geometry and texture features are effective for gender classification. This study aimed to combine these two features to improve the accuracy of gender classification. This research used the Viola-Jones and Orthogonal Difference Local Binary Pattern (OD-LBP) methods for feature extraction. The Viola-Jones algorithm faces issues in facial detection, leading to outliers in geometry features. At the same time, OD-LBP is a new descriptor capable of addressing pose, lighting, and expression variations. Therefore, this research attempts to utilize OD-LBP for gender classification. The dataset used was FERET, which contained various lighting variations, making OD-LBP suitable for addressing this challenge. Random Forest and Backpropagation were employed for classification. This research demonstrates that combining these two features is effective for gender classification using Backpropagation, achieving an accuracy of 93%. This confirms the superiority of the proposed method over single-feature extraction methods.

Face repairing based on transfer learning method with fewer training samples: application to a Terracotta Warrior with facial cracks and a Buddha with a broken nose

In this paper, a method based on transfer learning is proposed to recover the three-dimensional shape of cultural relics faces from a single old photo. It can simultaneously reconstruct the three-dimensional facial structure and align the texture of the cultural relics with fewer training samples. The UV position map is used to represent the three-dimensional shape in space and act as the output of the network. A convolutional neural network is used to reconstruct the UV position map from a single 2D image. In the training process, the human face data is used for pre-training, and then a small amount of artifact data is used for fine-tuning. A deep learning model with strong generalization ability is trained with fewer artifact data, and a three-dimensional model of the cultural relic face can be reconstructed from a single old photograph. The methods can train more complex deep networks without a large amount of cultural relic data, and no over-fitting phenomenon occurs, which effectively solves the problem of fewer cultural relic samples. The method is verified by restoring a Chinese Terracotta Warrior with facial cracks and a Buddha with a broken nose. Other applications can be used in the fields such as texture recovery, facial feature extraction, and three-dimensional model estimation of the damaged cultural relics or sculptures in the photos.

Quantum-inspired vortex search algorithm with deep neural networks for multi-pose facial expression recognition

The rapid expansion of artificial intelligence technologies has enabled machines to comprehend emotional intelligence. Among various indicators, facial expressions serve as an effective medium for understanding emotions. The concept of facial expression recognition (FER) relies heavily on the accurate and robust features available. Initially, the method of three-channel convolutional neural networks (TC-CNN) is adapted to extract facial features. However, only extracting the features is insufficient, the optimization of the extracted features is crucial to determining precise and robust features. This research work focuses on the optimization of the features using the quantum-inspired vortex search algorithm (QVSA). The QVSA integrates the attributes of Q-bits into the vortex search algorithm (VSA), optimizing the features by using the Q-bits to determine the vortex center on the Bloch sphere. The Q-bit attributes also improve the diversity of the features and help to avoid the premature convergence of the VSA. The final recognition of the facial expressions is performed using the deep neural network method of ResNet101v2. The experiments for facial expression recognition are performed on the datasets of RaFD and KDEF, which include different facial positions such as front pose, diagonal pose and profile pose. Performance comparisons demonstrate the effectiveness of the proposed system over state-of-the-art facial expression techniques.