Facial Landmark Features Research Articles

DEEPFAKE DETECTION USING TRANSFER LEARNING-BASED XCEPTION MODEL

Justification of the purpose of the research. In recent times, several approaches for face manipulation in videos have been extensively applied and availed to the public which makes editing faces in video easy for everyone effortlessly with realistic efforts. While beneficial in various domains, these methods could significantly harm society if employed to spread misinformation. So, it is also vital to properly detect whether a face has been distorted in a video series. To detect this deepfake, convolutional neural networks can be used in past works. However, it needs a greater number of parameters and more computations. So, to overcome these limitations and to accurately detect deepfakes in videos, a transfer learning-based model named the Improved Xception model is suggested. Obtained results. This model is trained using extracted facial landmark features with robust training. Moreover, the improved Xception model's detection accuracy is evaluated alongside ResNet and Inception, considering model loss, accuracy, ROC, training time, and the Precision-Recall curve. The outcomes confirm the success of the proposed model, which employs transfer learning techniques to identify fraudulent videos. Furthermore, the method demonstrates a noteworthy 5% increase in efficiency compared to current systems.

A Study on the Screening of Children at Risk for Developmental Disabilities Using Facial Landmarks Derived From a Mobile-Based Application

Objective Early detection and intervention of developmental disabilities (DDs) are critical to improving the long-term outcomes of afflicted children. In this study, our objective was to utilize facial landmark features from mobile application to distinguish between children with DDs and typically developing (TD) children.Methods The present study recruited 89 children, including 33 diagnosed with DD, and 56 TD children. The aim was to examine the effectiveness of a deep learning classification model using facial video collected from children through mobile-based application. The study participants underwent comprehensive developmental assessments, which included the child completion of the Korean Psychoeducational Profile-Revised and caregiver completing the Korean versions of Vineland Adaptive Behavior Scale, Korean version of the Childhood Autism Rating Scale, Social Responsiveness Scale, and Child Behavior Checklist. We extracted facial landmarks from recorded videos using mobile application and performed DDs classification using long short-term memory with stratified 5-fold cross-validation.Results The classification model shows an average accuracy of 0.88 (range: 0.78–1.00), an average precision of 0.91 (range: 0.75–1.00), and an average F1-score of 0.80 (range: 0.60–1.00). Upon interpreting prediction results using SHapley Additive exPlanations (SHAP), we verified that the most crucial variable was the nodding head angle variable, with a median SHAP score of 2.6. All the top 10 contributing variables exhibited significant differences in distribution between children with DD and TD (p<0.05).Conclusion The results of this study provide evidence that facial landmarks, utilizing readily available mobile-based video data, can be used to detect DD at an early stage.

Deep neural network-based emotion recognition using facial landmark features and particle swarm optimization

Relating specifically to human–computer interaction (HCI), computer vision research has placed a substantial emphasis on intelligent emotion recognition in recent years. The primary emphasis lies in investigating speech aspects and bodily motions, while the knowledge of recognizing emotions from facial expressions remains relatively unexplored. Automated facial emotion detection allows a machine to assess and understand a person's emotional state, allowing the system to predict intent by analyzing facial expressions. Therefore, this research provides a novel parameter selection strategy using swarm intelligence and a fitness function for intelligent recognition of micro emotions. This paper presents a novel method based on geometric visual representation obtained from facial landmark points. We employ the Deep Neural Networks (DNN) model to analyze the input features from the normalized angle and distance values derived from these landmarks. The results of the experiments show that Particle Swarm Optimization (PSO) worked very well by using only a few carefully chosen features. The method achieved a recognition success rate of 98.76% on the MUG dataset and 97.79% on the GEMEP datasets.

GFFT: Global-local feature fusion transformers for facial expression recognition in the wild

Facial expression recognition in the wild has become more challenging owing to various unconstrained conditions, such as facial occlusion and pose variation. Previous methods usually recognize expressions by holistic or relatively coarse local methods, but only capture limited features and are susceptible to be influenced. In this paper, we propose the Global–local Feature Fusion Transformers (GFFT) that is centered on cross-patch communication between features by self-attentive fusion. This method solves the problems of facial occlusion and pose variation effectively. Firstly, the Global Contextual Information Perception (GCIP) is designed to fuse global and local features, learning the relationship between them. Subsequently, the Facial Salient Feature Perception (FSFP) module is proposed to guide the fusion features to understand the key regions of facial features using facial landmark features to further capture face-related salient features. In addition, the Multi-scale Feature Fusion (MFF) is constructed to combine different stages of fusion features to reduce the sensitivity of the deep network to facial occlusion. Extensive experiments show that our GFFT outperforms existing state-of-the-art methods with 92.05% on RAF-DB, 67.46% on AffectNet-7, 63.62% on AffectNet-8, and 91.04% on FERPlus, demonstrating its effectiveness and robustness.

Visual Scene-Aware Hybrid and Multi-Modal Feature Aggregation for Facial Expression Recognition.

Facial expression recognition (FER) technology has made considerable progress with the rapid development of deep learning. However, conventional FER techniques are mainly designed and trained for videos that are artificially acquired in a limited environment, so they may not operate robustly on videos acquired in a wild environment suffering from varying illuminations and head poses. In order to solve this problem and improve the ultimate performance of FER, this paper proposes a new architecture that extends a state-of-the-art FER scheme and a multi-modal neural network that can effectively fuse image and landmark information. To this end, we propose three methods. To maximize the performance of the recurrent neural network (RNN) in the previous scheme, we first propose a frame substitution module that replaces the latent features of less important frames with those of important frames based on inter-frame correlation. Second, we propose a method for extracting facial landmark features based on the correlation between frames. Third, we propose a new multi-modal fusion method that effectively fuses video and facial landmark information at the feature level. By applying attention based on the characteristics of each modality to the features of the modality, novel fusion is achieved. Experimental results show that the proposed method provides remarkable performance, with 51.4% accuracy for the wild AFEW dataset, 98.5% accuracy for the CK+ dataset and 81.9% accuracy for the MMI dataset, outperforming the state-of-the-art networks.

Drowsiness Detection Based on Facial Landmark and Uniform Local Binary Pattern

When driving a vehicle, it is often challenging for someone to force his condition to keep driving even though in sleepy condition, thus causing a traffic accident. One of the characteristics of drowsy drivers is the eyes are closed for a certain period. This research proposes a system to detect drowsiness, thus can alert the drowsy driver. The first step is to detect the face using a Funnel-structured cascade algorithm. And then extract the facial landmark features on the face to get the eyes location. The features of eyes are extracted by using a Uniform Local Binary Pattern (ULBP) and the Eyes Aspect Ratio (EAR). EAR is the distance between points at eye landmarks. After the features have been extracted, the system classifies the eyes, whether closed or open by using Support Vector Machine (SVM) method. The system calculates the percentage of eye closure (PERCLOS) to detect drowsiness. Based on the experimental results, the proposed method yields the best accuracy of 95.5% and the optimal value of PERCLOS in drowsiness detection is greater than or equal to 60% with a period of 20 frames.

EAC-Net: Deep Nets with Enhancing and Cropping for Facial Action Unit Detection.

In this paper, we propose a deep learning based approach for facial action unit (AU) detection by enhancing and cropping regions of interest of face images. The approach is implemented by adding two novel nets (a.k.a. layers): the enhancing layers and the cropping layers, to a pretrained convolutional neural network (CNN) model. For the enhancing layers (noted as E-Net), we have designed an attention map based on facial landmark features and apply it to a pretrained neural network to conduct enhanced learning. For the cropping layers (noted as C-Net ), we crop facial regions around the detected landmarks and design individual convolutional layers to learn deeper features for each facial region. We then combine the E-Net and the C-Net to construct a so-called Enhancing and Cropping Net (EAC-Net), which can learn both features enhancing and region cropping functions effectively. The EAC-Net integrates three important elements, i.e., learning transfer, attention coding, and regions of interest processing, making our AU detection approach more efficient and more robust to facial position and orientation changes. Our approach shows a significant performance improvement over the state-of-the-art methods when tested on the BP4D and DISFA AU datasets. The EAC-Net with a slight modification also shows its potentials in estimating accurate AU intensities. We have also studied the performance of the proposed EAC-Net under two very challenging conditions: (1) faces with partial occlusion and (2) faces with large head pose variations. Experimental results show that (1) the EAC-Net learns facial AUs correlation effectively and predicts AUs reliably even with only half of a face being visible, especially for the lower half; (2) Our EAC-Net model also works well under very large head poses, which outperforms significantly a compared baseline approach. It further shows that the EAC-Net works much better without a face frontalization than with face frontalization through image warping as pre-processing, in terms of computational efficiency and AU detection accuracy.

Landmark localization on 3D/4D range data using a shape index-based statistical shape model with global and local constraints

In this paper we propose a novel method for detecting and tracking facial landmark features on 3D static and 3D dynamic (a.k.a. 4D) range data. Our proposed method involves fitting a shape index-based statistical shape model (SI-SSM) with both global and local constraints to the input range data. Our proposed model makes use of the global shape of the facial data as well as local patches, consisting of shape index values, around landmark features. The shape index is used due to its invariance to both lighting and pose changes. The fitting is performed by finding the correlation between the shape model and the input range data. The performance of our proposed method is evaluated in terms of various geometric data qualities, including data with noise, incompletion, occlusion, rotation, and various facial motions. The accuracy of detected features is compared to the ground truth data as well as to start of the art results. We test our method on five publicly available 3D/4D databases: BU-3DFE, BU-4DFE, BP4D-Spontaneous, FRGC 2.0, and Eurecom Kinect Face Dataset. The efficacy of the detected landmarks is validated through applications for geometric based facial expression classification for both posed and spontaneous expressions, and head pose estimation. The merit of our method is manifested as compared to the state of the art feature tracking methods.

RECONSTRUCT 3D HUMAN FACE USING TWO ORTHOGONAL IMAGES

As the 3D human face reconstruction is becoming very popular in recent times, it attracts many researchers. Construction of 3D human face using only two orthogonal images and twelve landmark features are the main context of the proposed approach. For3D object modeling, the Open Graphics Library (OpenGL) is used as the platform through which modeling, modification and rendering is performed on the morphable model. The proposed approach trails through semi-automatic identification of the facial landmark features, calculation of the 3D coordinates of human face, morphable model construction in OpenGL, reshaping of the morphable model and rendering of the morphable model. The facial landmark identification is semiautomatic method, as the module requires the manual interaction for marking the facial landmarks on the image. The reshaping of morphable model is required as the morphable model does not fit to the actual face in most of the cases. The morphable model is reshaped by calculating root mean square (RMS) error of face coordinates. The rendering process does not require a wide screen image because the approach performs rendering using input front face image and side face image as textures. Applications of this research help to overcome the disputes of fields like crime detection, 3D game characterization, the ornaments exhibition and in the area of medical technology like plastic surgery.