Facial Landmarks Research Articles

Generalized face forgery detection with self-supervised face geometry information analysis network

The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel Self-supervised Face Geometry Information Analysis Network (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.

DualFaceNet: augmentation consistency for optimal facial landmark detection and face mask classification

<p>In an era where face masks are commonplace, facial recognition faces new challenges and opportunities. This study introduces DualFaceNet (DFN), a cutting-edge neural network that efficiently combines facial landmark detection with mask classification. Benefiting from multi-task learning (MTL) and enhanced with a unique consistency loss, DFN outperforms traditional single-task models. Tests using the reputable 300W dataset and a face mask dataset showcase DFN’s strengths: a significant reduction in landmark error to 5.42 and an increase in mask classification accuracy to 92.59%. These results highlight the potential of integrating MTL and custom loss functions in facial recognition. As face masks continue to be globally essential, DFN’s integrated approach offers a fresh perspective in facial recognition studies. Furthermore, DFN paves the way for adaptive facial recognition systems, emphasizing the adaptability needed in our current era.</p>

Quarantine Order Violators System using Face Recognition (FACID)

Face recognition technology is commonly utilized for user authentication and verification by analysing a digital image of a person's face and matching it against a database of faces for identification purposes. The COVID-19 pandemic has led to mandatory quarantines and the use of quarantine bracelets for some individuals, which can be time-consuming and require a lot of effort. Face recognition technology can help raise awareness about the current situation and alleviate some of the burden associated with quarantine measures. The aim of this research is to concentrate on the face recognition of individuals with a history of quarantine, as a measure to prevent the spread of COVID-19 in educational institutions like universities, colleges, and schools. This research study concentrates on individuals with a history of quarantine orders. The system will employ the Histograms of Oriented Gradients (HOG) algorithm for face detection. Additionally, the system will utilize the Face Landmark Algorithm to compare the 128-d vector with images stored locally. The system will make use of the Helen face collection dataset for its data requirements. The aim of this study is to explore and identify techniques for detecting individuals who violate quarantine measures and issuing notifications through the proposed system. By implementing this system, it could contribute to creating a more secure environment within the educational institution and potentially reduce the spread of the virus.

Accuracy is not enough: a heterogeneous ensemble model versus FGSM attack

AbstractIn this paper, based on facial landmark approaches, the possible vulnerability of ensemble algorithms to the FGSM attack has been assessed using three commonly used models: convolutional neural network-based antialiasing (A_CNN), Xc_Deep2-based DeepLab v2, and SqueezeNet (Squ_Net)-based Fire modules. Firstly, the three individual deep learning classifier-based Facial Emotion Recognition (FER) classifications have been developed; the predictions from all three classifiers are then merged using majority voting to develop the HEM_Net-based ensemble model. Following that, an in-depth investigation of their performance in the case of attack-free has been carried out in terms of the Jaccard coefficient, accuracy, precision, recall, F1 score, and specificity. When applied to three benchmark datasets, the ensemble-based method (HEM_Net) significantly outperforms in terms of precision and reliability while also decreasing the dimensionality of the input data, with an accuracy of 99.3%, 87%, and 99% for the Extended Cohn-Kanade (CK+), Real-world Affective Face (RafD), and Japanese female facial expressions (Jaffee) data, respectively. Further, a comprehensive analysis of the drop in performance of every model affected by the FGSM attack is carried out over a range of epsilon values (the perturbation parameter). The results from the experiments show that the advised HEM_Net model accuracy declined drastically by 59.72% for CK + data, 42.53% for RafD images, and 48.49% for the Jaffee dataset when the perturbation increased from A to E (attack levels). This demonstrated that a successful Fast Gradient Sign Method (FGSM) can significantly reduce the prediction performance of all individual classifiers with an increase in attack levels. However, due to the majority voting, the proposed HEM_Net model could improve its robustness against FGSM attacks, indicating that the ensemble can lessen deception by FGSM adversarial instances. This generally holds even as the perturbation level of the FGSM attack increases.

Hard and Soft Tissue Facial Landmarks for Mandibular Angle Reduction: A Clinical Study.

Square faces, which are influenced by genetic factors and structural features, are considered undesirable among the Asian population. Surgical interventions, such as mandibular angle reduction, aim to alter these characteristics, though complications may arise. We aimed to investigate the morphology of the mandibular angle and masseter muscle thickness using computed tomography (CT) and to analyze hard and soft tissue correlations to enhance surgical outcomes for patients with square faces. This retrospective clinical study included 100 Taiwanese patients aged 18-50 years. CT was used to analyze key clinical parameters, including bilateral mandibular width, mandibular divergence angle, ramus height, distance from the mandibular angle to the inferior alveolar nerve (IAN), and the thickness of the masseter muscle. Significant correlations were noted between the patients' physical height and weight, mandibular width, ramus height, masseter thickness, and distance from the angle to the IAN. Males exhibited a significantly longer and thicker ramus height (66.48 ± 4.28 mm), greater masseter thickness (15.46 ± 2.35 mm), and greater safety range for mandibular angle reduction surgery (18.35 ± 3.19 mm) (p < 0.00008). Significant correlations were observed among all parameters, except between mandibular width and gonial angle and the distance from the angle to the IAN and between mandibular divergence and masseter muscle thickness (p > 0.1). Our study highlighted the complex interplay among factors that contribute to square facial morphology. Careful preoperative assessments and customized surgical planning are essential for addressing this multifaceted clinical challenge.

Barrier Function to Skin Elasticity in Talking Head

In this paper, we target the problem of generating facial expressions from a piece of audio. This is challenging since both audio and video have inherent characteristics that are distinct from the other. Some words may have identical lip movements, and speech impediments may prevent lip-reading in some individuals. Previous approaches to generating such a talking head suffered from stiff expressions. This is because they focused only on lip movements and the facial landmarks did not contain the information flow from the audio. Hence, in this work, we employ spatio-temporal independent component analysis to accurately sync the audio with the corresponding face video. Proper word formation also requires control over the face muscles that can be captured using a barrier function. We first validated the approach on the diffusion of salt water in coastal areas using a synthetic finite element simulation. Next, we applied it to 3D facial expressions in toddlers for which training data is difficult to capture. Prior knowledge in the form of rules is specified using Fuzzy logic, and multi-objective optimization is used to collectively learn a set of rules. We observed significantly higher F-measure on three real-world problems.

Learning facial structural dependency in 3D aligned space for face alignment

Facial structure's statistical characteristics offer pivotal prior information in facial landmark prediction, forming inter-dependencies among different landmarks. Such inter-dependencies ensure that predictions adhere to the shape distribution typical of natural faces. In challenging scenarios like occlusions or extreme facial poses, this structure becomes indispensable, which can help to predict elusive landmarks based on more discernible ones. While current deep learning methods do capture these landmark dependencies, it's often an implicit process heavily reliant on vast training datasets. We contest that such implicit modeling approaches fail to manage more challenging situations. In this paper, we propose a new method that harnesses the facial structure and explicitly explores inter-dependencies among facial landmarks in an end-to-end fashion. We propose a Structural Dependency Learning Module (SDLM). It uses 3D face information to map facial features into a canonical UV space, in which the facial structure is explicitly 3D semantically aligned. Besides, to explore the global relationships between facial landmarks, we take advantage of the self-attention mechanism in the image and UV spaces. We name the proposed method Facial Structure-based Face Alignment (FSFA). FSFA reinforces the landmark structure, especially under challenging conditions. Extensive experiments demonstrate that FSFA achieves state-of-the-art performance on the WFLW, 300W, AFLW, and COFW68 datasets.

A Survey an Optimized Dense CNN Model for Recognizing Deepfake Images

Deepfake detection relies on a deep learning model. Deepfake content is generated using artificial intelligence and machine learning techniques to swap one person’s face with another’s in images. These altered images are certain to have a significant impact on society. Deepfakes utilize advanced technologies such as machine learning (ML) and deep learning (DL) to develop automated techniques for generating deceptive content. The model is trained using the DFDC (Deepfake Detection Challenge) dataset, which includes 1, 00,000 videos comprising both real and fake content. This survey examines recent progress in deep learning techniques for detecting deepfake images, highlighting their growing significance in today’s digital landscape. The paper outlines the issues arising from deepfakes, explores the techniques for generating deceptive content, and analyzes various deep learning models employed in deepfake detection, such as convolutional neural networks (CNN’s), MTCNN (Multi-task CNN), and Facial Landmark. It emphasizes the ongoing battle between the progression of deepfakes and the evolution of detection methods, underscoring the need for advanced and flexible neural network architectures to effectively curb the dissemination of deceptive information.

Exploring Attentive User Interface Input via Raspberry Pi, Based on Face Landmark Detection, Eye Open-Closed Detection and Head Movements Detection

Exploring Attentive User Interface Input via Raspberry Pi, Based on Face Landmark Detection, Eye Open-Closed Detection and Head Movements Detection

Three-dimensional computer navigation in the reconstruction of complex unilateral orbital fractures: evaluation and review of applications.

The eyes are the central aesthetic unit of the face. Maxillofacial trauma can alter facial proportions and affect visual function with varying degrees of severity. Conventional approaches to reconstruction have numerous limitations, making the process challenging. The primary objective of this study was to evaluate the application of three-dimensional (3D) navigation in complex unilateral orbital reconstruction. A prospective cohort study was conducted over 19 months (January 2020 to July 2021), with consecutive enrollment of 12 patients who met the inclusion criteria. Each patient was followed for a minimum period of 6 months. The principal investigator carried out a comparative analysis of several factors, including fracture morphology, orbital volume, globe projection, diplopia, facial morphic changes, lid retraction, and infraorbital nerve hypoesthesia. Nine patients had impure orbital fractures, while the remainder had pure fractures. The median orbital volume on the normal side (30.12 cm3; interquartile range [IQR], 28.45-30.64) was comparable to that of the reconstructed orbit (29.67 cm3; IQR, 27.92-31.52). Diplopia improved significantly (T(10) = 2.667, p = 0.02), although there was no statistically significant improvement in globe projection. Gross symmetry of facial landmarks was achieved, with comparable facial width-to-height ratio and palpebral fissure lengths. Two patients reported infraorbital hypoesthesia at presentation, which persisted at the 6-month follow-up. Additionally, five patients developed lower lid retraction (1-2 mm), and one experienced implant impingement at the infraorbital border. Our study provides level II evidence supporting the use of 3D navigation to improve surgical outcomes in complex orbital reconstruction.

A Survey Study of the 3D Facial Landmark Detection Techniques Used as a Screening Tool for Diagnosis of the Obstructive Sleep Apnea Syndrome.

Obstructive Sleep Apnea (OSA) is a common disorder affecting both adults and children. It is characterized by repeated episodes of apnea (stopped breathing) and hypopnea (reduced breathing), which result in intermittent hypoxia. We recognize pediatric and adult OSA, and this paper focuses on pediatric OSA. While adults often suffer from daytime sleepiness, children are more likely to develop behavioral abnormalities. Early diagnosis and treatment are important to prevent negative effects on children's development. Without the treatment, children may be at increased risk of developing high blood pressure or other heart problems. The gold standard for OSA diagnosis is the polysomnography (sleep study) PSG performed at a sleep center. Not only is it an expensive procedure, but it can also be very stressful, especially for children. Patients have to stay at the sleep center during the night. Therefore, screening tools are very important. Multiple studies have shown that OSA screening tools can be based on facial anatomical landmarks. Anatomical landmarks are landmarks located at specific anatomical locations. For the purpose of the screening tool, a specific list of anatomical locations needs to be identified. We are presenting a survey study of the automatic identification of these landmarks on 3D scans of the patient's head. We are considering and comparing both knowledge-based and AI-based identification techniques, with a focus on the development of the automatic OSA screening tool.

Do executive facial trustworthiness have impact on IPO underpricing in the Indonesia stock exchange?

PurposeThis study aims to investigate the impact of facial-based perceived trustworthiness on stock valuation, particularly, in the initial public offering (IPO). IPO settings provide the opportunity to investigate whether information asymmetry resulting from company newness in the market would influence the incorporation of soft information in the form of executive facial trustworthiness in stock valuation.Design/methodology/approachWe use a recent machine learning algorithm to detect facial landmarks and then calculate a composite facial trustworthiness measure using several facial features that have previously been observed in neuroscience and psychological studies to be the most determining factor of perceived trustworthiness. We then regress the facial trustworthiness of IPO firm executives to IPO underpricing.FindingsUtilizing machine learning algorithms, we find that the facial trustworthiness of the company executive negatively impacts the extent of IPO underpricing. This result implies that investors incorporate the facial trustworthiness of company executives into stock valuation. The IPO underpricing also shows that the cost of equity is higher when perceived trustworthiness is low. With regard to the higher information asymmetry in IPO transactions, such a negative impact implies the role of facial trustworthiness in alleviating information asymmetry.Originality/valueThis study provides evidence of the impact of top management personal characteristics on firms’ financial transactions in the Indonesian context. From the perspective of investors and other fund providers, this study shows evidence that heuristics still play an important role in financial decision-making. This is also an indication of investor reliance on soft information. Our research method also provides a new opportunity for the use of machine-learning algorithms in processing non-conventional types of data in finance research, which is still relatively rare in emerging markets like Indonesia. To the best of our knowledge, our study is the first to use personalized measures of trust generated through machine-learning algorithms in IPO settings in Indonesia.

A Hybrid Cloud Approach for Secure Authorized Deduplication System Using Facial Authentication

Over the use of peer-to-peer encryption, businesses are able to establish private communication channels between devices or parts of devices, ensuring that no intermediary devices are able to decipher important data as it travels over the network. To safeguard sensitive information like credit card details, PIN numbers, and passwords, privacy is essential. Rather of being handed down orally, this kind of knowledge may be exchanged digitally using messaging programs such as WhatsApp or Facebook Messenger.Most of these apps use what is known as peer-to-peer encryption, a kind of two-way communication in which no one other than the two parties involved can decipher the communications transmitted and received. It is necessary to remember the master password in order to access the password safe, which stores all the passphrases in one account. Other malevolent bots and hackers are prompted to plot an assault and attempt to breach the system by this strategy.Therefore, we provide an encryption technique in our suggested work that relies on the authentic user's facial landmarks. Since it is very difficult, if not impossible, to replicate all of the face characteristics in order to fool the authentication system, this method is extremely strong. To determine the legitimate user's identity, we use the Localised Binary Pattern method in conjunction with the Gabor Wavelet on the locality preserving projection model. Both the legitimacy on the peer-to-peer network and the encryption of data communicated across the network channel are supported by the face structure that has been extracted. As a result, this forms a very strong component for the authentication and encryption of sent communications. Key Words: peer-to-peer, system, encryption, authentication, breach, transmitted

A Study on Facial Landmark Detection and Image Processing

A Study on Facial Landmark Detection and Image Processing

Facial micro-expression recognition based on motion magnification network and graph attention mechanism

Micro-expression is extensively studied due to their ability to fully reflect individuals' genuine emotions. However, accurate micro-expression recognition is a challenging task due to the subtle motion of facial muscle. Therefore, this paper introduces a Graph Attention Mechanism-based Motion Magnification Guided Micro-Expression Recognition Network (GAM-MM-MER) to amplify delicate muscle motions and focus on key facial landmarks. First, we propose a Swin Transformer-based network for micro-expression motion magnification (ST-MEMM) to enhance the subtle motions in micro-expression videos, thereby unveiling imperceptible facial muscle movements. Then, we propose a graph attention mechanism-based network for micro-expression recognition (GAM-MER), which optimizes facial key area maps and prioritizes adjacent nodes crucial for mitigating the influence of noisy neighbors, while attending to key feature information. Finally, experimental evaluations conducted on the CASME II and SAMM datasets demonstrate the high accuracy and effectiveness of the proposed network compared to state-of-the-art approaches. The results of our network exhibit significant superiority over existing methods. Furthermore, ablation studies provide compelling evidence of the robustness of our proposed network, substantiating its efficacy in micro-expression recognition.

A Comprehensive Assessment of Soft-tissue Sagging after Zygoma Reduction Surgery through Artificial Intelligence Analysis.

Overdevelopment of zygomatic bones often results in protrusion and flaring of the midfacial region. This makes the face appear squarer than the more favorable oval shape. Therefore, zygoma reduction surgery has become a commonly performed procedure in patients seeking to obtain an ideal facial shape. Facial soft-tissue ptosis is one of the main complications of zygoma reduction surgery. Previously, the evaluation of cheek soft-tissue ptosis was subjectively based on patients and surgeons. Our study aimed to provide an objective evaluation of soft-tissue sagging in the cheek region after zygoma reduction surgery using artificial intelligence (AI). We used AI to evaluate cheek sagging in a series of patients who underwent zygoma reduction surgery. We used four methods: tracking facial landmarks, detecting changes in the cheek curvature, and examining changes in the nasolabial fold and marionette lines. Then, the obtained numerical results were assessed for statistically significant differences using statistical validation methods. Use of AI with the four methods demonstrated no statistically significant differences between the pre- and postsurgery evaluations. AI analysis demonstrated that soft-tissue ptosis did not occur in our series of patients. AI offers objective evaluation for both patients and doctors. Future research could build on this application to examine various influencing factors and develop new tools using machine learning to evaluate and predict the extent of cheek sagging in patients before surgery.

Facial Expression Analysis Using Multi-label Classification

Emotions form the reflection of an individual's mental state. Automatic expression analysis and its recognition based on their facial landmarks are of critical use, particularly to understand the emotional states of physically disabled individuals like autistic children, deaf, dumb, and bedridden individuals. This approach helps in interpreting human emotions, which is crucial for designing computational models and interfaces that cater to their needs. While traditional research has focused on six basic emotion categories (happiness, surprise, sadness, anger, fear, disgust), your work expands this by exploring compound emotion categories. Compound emotions combine these basic categories to create nuanced emotional states. The research utilizes the CFEE_Database_230 dataset of facial expressions captured for analysis and training purposes. The proposed methodology has the following three steps: Analyze the dataset and extract the region of interest (ROI), Extract various statistical and potentially discriminating features and apply a multi-label classification approach to categorize sets of emotions. This involves comparing the feature values across different emotion classes and assigning appropriate labels, particularly focusing on compound emotion categories. Also, the research employs different classifiers and metrics to evaluate the effectiveness of the model. After applying the classification methods, the results are analyzed using various metrics to assess the accuracy and effectiveness of emotion recognition based on facial landmarks. Based on the findings, the Binary Relevance Method yielded the best performance with Mean Average Precision and hamming Loss of 0.7227±0.0344 and 0.1906±0.0227, respectively. Overall, the work contributes to advancing automatic emotion recognition by considering a broader range of emotional categories beyond the traditional basics. This is particularly beneficial for populations such as the physically disabled and autistic children, where traditional communication methods might be limited or challenging.

Managing Long-term Orofacial Contractures and Microstomia Through Intraoral Stretching.

Microstomia and orofacial contractures continue to garner interest regarding the effectiveness of treatment methodologies utilized to impact functional change. Oral splints are an accepted tool in the management of microstomia. However, the concepts of which oral splints to use, timing of initiation, and duration of treatment have not gained a consensus. This article reviews approaches to oral splinting and an alternative intraoral approach using splints designed to provide a graded, low load, multidirectional, and prolonged stretch specifically in facial burn survivors including those with mature scars. Two participants participated in a trial using oral splints placed inside the mouth at established contracture points. Participants were requested to use the splints for 1 h twice daily. Participants were photographed weekly producing 9 facial expressions, and distance between 13 facial landmarks was measured to evaluate change in tissue length. Numerical changes observed from beginning to end of the trial indicate positive and negative alterations, signifying lengthening or shortening of tissue. Negative changes denote reduction in distance between endpoints, while positive changes signify an increase. Participants verbalized functional improvements in oral motor and psychosocial function posttreatment. To date, oral splints can be custom fabricated for each individual patient. However, few oral splints are created to provide multidirectional stretch focusing on problem areas across the mid and lower face. The intraoral splints and regimen described here have the capability of providing a stretching intervention that could be applicable in various stages of burn recovery.

FaceMotionPreserve: a generative approach for facial de-identification and medical information preservation

Telemedicine and video-based diagnosis have raised significant concerns regarding the protection of facial privacy. Effective de-identification methods require the preservation of diagnostic information related to normal and pathological facial movements, which play a crucial role in the diagnosis of various movement, neurological, and psychiatric disorders. In this work, we have developed FaceMotionPreserve , a deep generative model-based approach that transforms patients’ facial identities while preserving facial dynamics with a novel face dynamic similarity module to enhance facial landmark consistency. We collected test videos from patients with Parkinson’s disease recruited via telemedicine for evaluation of model performance and clinical applicability. The performance of FaceMotionPreserve was quantitatively evaluated based on neurologist diagnostic consistency, critical facial behavior fidelity, and correlation of general facial dynamics. In addition, we further validated the robustness and advancements of our model in preserving medical information with clinical examination videos from a different cohort of patients. FaceMotionPreserve is applicable to real-time integration, safeguarding facial privacy while retaining crucial medical information associated with facial movements to address concerns in telemedicine, and facilitating safer and more collaborative medical data sharing.

Precise facial landmark detection by Dynamic Semantic Aggregation Transformer

At present, deep neural network methods have played a dominant role in face alignment field. However, they generally use predefined network structures to predict landmarks, which tends to learn general features and leads to mediocre performance, e.g., they perform well on neutral samples but struggle with faces exhibiting large poses or occlusions. Moreover, they cannot effectively deal with semantic gaps and ambiguities among features at different scales, which may hinder them from learning efficient features. To address the above issues, in this paper, we propose a Dynamic Semantic-Aggregation Transformer (DSAT) for more discriminative and representative feature (i.e., specialized feature) learning. Specifically, a Dynamic Semantic-Aware (DSA) model is first proposed to partition samples into subsets and activate the specific pathways for them by estimating the semantic correlations of feature channels, making it possible to learn specialized features from each subset. Then, a novel Dynamic Semantic Specialization (DSS) model is designed to mine the homogeneous information from features at different scales for eliminating the semantic gap and ambiguities and enhancing the representation ability. Finally, by integrating the DSA model and DSS model into our proposed DSAT in both dynamic architecture and dynamic parameter manners, more specialized features can be learned for achieving more precise face alignment. It is interesting to show that harder samples can be handled by activating more feature channels. Extensive experiments on popular face alignment datasets demonstrate that our proposed DSAT outperforms state-of-the-art models in the literature. Our code is available at https://github.com/GERMINO-LiuHe/DSAT.