Facial Occlusion Research Articles

Occluded facial expression recognition based on deep learning

Abstract. When employing Convolutional Neural Networks (CNNs) for facial expression recognition, several challenges are often encountered, such as facial occlusions, the limited size of reliable expression datasets, and inadequate precision in recognition outcomes. This paper preprocesses the dataset to enhance its reliability. By leveraging data synthesis and augmentation techniques, it employs a method of randomly generating occlusion blocks to integrate and expand the dataset. Based on the ResNet-18 network, the model is optimized by incorporating an attention mechanism, thereby improving the network's precision and robustness in recognizing facial expressions.

LW-DCGAN: a lightweight deep convolutional generative adversarial network for enhancing occluded face recognition

LW-DCGAN: a lightweight deep convolutional generative adversarial network for enhancing occluded face recognition

A Review of Traditional Methods and Deep Learning for Face Recognition

Face recognition technology is a key research direction in artificial intelligence and computer vision. It first originated in the 1960s, when Bledsoe et al. proposed and studied computer programming based on face recognition. It is divided into three steps: face detection, feature extraction, and face verification. This paper briefly introduces the wide application of face recognition in various fields such as vulnerable groups, analyses its characteristics and progress from the viewpoints of traditional learning and deep learning, and focuses on the methods based on face feature extraction (including geometric features, texture features, local features, etc.). From the previous studies appear some problems connected to environmental changes, such as light change, facial occlusion, posture and privacy leakage, etc. This paper will give the possible reasons for these problems and give the possible development direction of the future by exploring the current face recognition technology to provide a reference for the related research of face recognition technology.

New Extracted Features to Recognize Faces Effected by Occlusions and Common Variations

Face recognition from non-identical face photos is a prominent area of research in pattern recognition and computer vision. Existing face recognition systems struggle with diverse changes like lighting conditions, expressions, and facial occlusions. This paper proposes a new Face Recognition (FR) approach that combines the Elastic Bunch Graph Matching (EBGM) approach with the greedy algorithm to automatically identify face landmarks. The proposed approach independently selects each optimal landmark of face image from different corresponding face images where the corresponding landmark of corresponding face image which achieves the best similarity is used rather than using one or at most two corresponding face images and computing the average between both. The locations of corresponding landmarks can be displaced to achieve maximum similarity with optimal landmarks. This proposed approach demonstrates improved recognition performance compared to contemporary face recognition methods. It effectively handles changing ratios of face parts and can recognize faces even with increasing occlusion sizes.

DEEP Q-NETWORK (DQN) PARTIALOCCLUSION SEGMENTATION AND BACKTRACKING SEARCH OPTIMIZATION ALGORITHM (BSOA) WITH OPTICAL FLOW RECONSTRUCTION FOR FACIAL EXPRESSION EMOTION RECOGNITION

Video facial expression recognition (FER) has garnered a lot of attention recently and is helpful for several applications. Although many algorithms demonstrate impressive performance in a controlled environment without occlusion, identification in the presence of partial facial occlusion remains a challenging issue. Solutions based on reconstructing the obscured area of the face have been suggested as a way to deal with occlusions. These options mostly rely on the face’s shape or texture. Nonetheless, the resemblance in facial expressions among individuals appears to be a valuable advantage for the reconstruction. For semantic segmentation based on occlusions, Reinforcement Learning (RL) is introduced as the initial stage. From a pool of unlabeled data, an agent learns a policy to choose a subset of tiny informative image patches to be tagged instead of full images. In the second stage, a trained Backtracking Search Algorithm (BSA) is used to rebuild optical flows that have been distorted by the occlusion. On obtaining optical flows estimated from occluded facial frames, AEs restore optical flows of occluded regions. These recovered optical flows become inputs to anticipate classes f expressions. Optical flux reconstructions then classify stages. This study evaluates classification model’s performances for face expression identification based on Very Deep Convolution Networks (VGGNet). Furthermore, it produces more accurate confusion matrices and proposes approaches for the KMU-FED and CK+ databases, respectively. The results are evaluated using metrics including recall, f-measure, accuracy, and precision.

Robust Face Recognition System Based on Deep Facial Feature Extraction and Machine Learning

Facial Recognition is a common challenge in research for security applications. Wide-ranging applications of this technology include biometrics, security data, access to controlled locations, smart cards, and surveillance systems. A practical approach to identifying individuals depends on factors such as partial face occlusion, illumination, age, expression, makeup, and poses. These complex face recognition variables affect most face recognition systems. In this paper, we propose a deep transfer learning for facial features based on the pose and illumination variations of celebrities’ faces in a free environment for face recognition. The new approach that combines deep learning methods for feature extraction with machine learning classification methods. Also, studies the performance of the pre-trained model (VGGFace2 with ResNet-50 model weights) for feature extraction with a Multilayer Perceptron Classifier (MLP), Decision Tree, and Bootstrap Aggregation (Bagging) to perform classification. The convolution neural network has lately achieved excellent advancement in facial recognition. The outcome indicated that VGGFace2 with MLP obtained more precision and provided the best results (Accuracy, F-measure, Recall, and Precision). The results for all metric for the proposed model is 99%. The benchmark for this model is a dataset of 105 people’s faces.

EXPRESS: The impact of partially covered faces on trust attribution, sharing resources and perceived fairness of one's own choices in Ultimatum Game.

Covered faces has been linked with impaired emotion recognition, yet it is entirely unexplored how an occlusion due to face masks may affect individuals' behavior in economic decisions. Across two studies, we explored whether partially covered faces (due to mask wearing or a horizontal black bar), and emotion displayed by the Responder influence peoples' sharing behavior in the Ultimatum Game and perceived fairness of one's proposal.Study 1 showed participants were more willing to equally share their resources with a happy face (compared to a neutral one). In addition, they were more willing to make a fair proposal when the person displayed was not wearing a face mask. Our results also provide evidence that, when people had to judge how fair was their proposal, participants rated a fair proposal as fairer when when Responders showed happy faces without masks, while unfair proposals were rated as fairer with happy masked faces; similarly, angry faces led to fairer ratings for fair offers without masks and unfair offers with masks. Study 2 partially confirmed previous results, highlighting how a simple occlusion on the face does not have a direct effect on the proposal, but moderates the effect of the displayed emotions.These findings indicate that social interactions might be affected by face occlusion, especially when it is represented by a face mask. Indeed, people might judge the same behavior in different ways based on the fact that their counterpart has a partially covered face.

Beyond the Whole: Reduced Empathy for Masked Emotional Faces Is Not Driven by Disrupted Configural Face Processing.

Sharing of emotional states is reduced for individuals wearing face coverings, but the mechanism behind this reduction remains unknown. Here, we investigated if face occlusion by masks reduces empathy by disrupting configural processing of emotional faces. Participants rated their empathy for happy and neutral faces which were presented in upright or inverted orientation and wore opaque, clear, or no face masks. Empathy ratings were reduced for masked faces (opaque or clear) as well as for inverted faces. Importantly, face inversion disrupted empathy more for faces wearing opaque masks relative to those wearing clear or no masks, which stands in contrast to the predictions generated by the classic configural processing models. We discuss these data within the context of classic and novel configural face perception models, and highlight that studying inverted occluded faces presents an informative case worthy of further investigation.

Lower empathy for face mask wearers is not explained by observer's reduced facial mimicry.

Facial occlusion alters social processes that rely on face visibility, including spontaneous mimicry of emotions. Given that facial mimicry of emotions is theorized to play an important role in how we empathize or share emotions with others, here we investigated if empathy was reduced for faces wearing masks because masks may reduce the ability to mimic facial expressions. In two preregistered experiments, participants rated their empathy for faces displaying happy or neutral emotions and wearing masks or no masks. We manipulated mimicry by either blocking mimicry with observers holding a pen in between their teeth (Experiment 1) or by producing a state of constant congruent mimicry by instructing observers to smile (Experiment 2). Results showed reduced empathy ratings for masked faces. Mimicry overall facilitated empathy, with reduced empathy ratings when mimicry was blocked and higher empathy ratings when it was instructed. However, this effect of mimicry did not vary with mask condition. Thus, while observers were impaired in sharing emotions with masked faces, this impairment did not seem to be explained by a reduction in facial mimicry. These results show that mimicry is an important process for sharing emotions, but that occluding faces with masks reduces emotion sharing via a different mechanism.

Comprehensive comparison between vision transformers and convolutional neural networks for face recognition tasks

This paper presents a comprehensive comparison between Vision Transformers and Convolutional Neural Networks for face recognition related tasks, including extensive experiments on the tasks of face identification and verification. Our study focuses on six state-of-the-art models: EfficientNet, Inception, MobileNet, ResNet, VGG, and Vision Transformers. Our evaluation of these models is based on five diverse datasets: Labeled Faces in the Wild, Real World Occluded Faces, Surveillance Cameras Face, UPM-GTI-Face, and VGG Face 2. These datasets present unique challenges regarding people diversity, distance from the camera, and face occlusions such as those produced by masks and glasses. Our contribution to the field includes a deep analysis of the experimental results, including a thorough examination of the training and evaluation process, as well as the software and hardware configurations used. Our results show that Vision Transformers outperform Convolutional Neural Networks in terms of accuracy and robustness against distance and occlusions for face recognition related tasks, while also presenting a smaller memory footprint and an impressive inference speed, rivaling even the fastest Convolutional Neural Networks. In conclusion, our study provides valuable insights into the performance of Vision Transformers for face recognition related tasks and highlights the potential of these models as a more efficient solution than Convolutional Neural Networks.

In Vitro Study Of Molar Tooth With Endodontic Treatment At Endocrown Successful Prosthetic Rehabilitation

Background: Side teeth, having a decisive role for mastication, require special attention after endodontic treatment. The rehabilitation of these teeth includes the analysis of current restoration techniques, materials used, as well as aesthetic and functional impact on the patient. Through a multidisciplinary approach, it aims not only to recover dental function, but also to improve the quality of life of the patient by restoring a healthy and harmonious smile. Materials and Methods: In this study, we conducted tests on 30 extracted teeth that had whole or partially whole dental crowns, the condition of selecting the tooth in the study being at least 3 mm of dental tissue present, and, necessary for the subsequent prosthetic preparation for endocrown prosthetics. The 30 teeth were divided into 6 batches, the control group (A) being made up only of integral teeth that were milling for covering crowns. The other teeth were divided into 5 batches (B-F) of 5 teeth, the teeth were endodontically treated and milling for prosthetic rehabilitation with endocrown. Each lot (B-F) benefited from a preparation of coronoradicular cylindrical shape of different depths between 0.5-2,5mm, on the occlusal face of the tooth in the endodontic access area, in order to increase the surface of the retention coronary dental tissues for the stability of the prosthetic crowns of the coating. Each tooth in A-F batches has benefited from 3 crowns made of the same material through 3 different technologies. Results: The values recorded by the A-lot, rehabilitated with crowns of sheathing on integral teeth, recorded an average value of the crown extraction force on tooth of 25.42±0.55 for CAD-CAM technology, for crowns made by baro-thermo-polymerization technique an average of 24,14±0,37, and for hot polymerization technique an average of 23,66±0,37. For B-F batches, the closest values for the extraction of endocorons were obtained by lot D with a preparation in the corono-radicular axe of 1,5mm, both for the CAD-CAM technology with a average of 25,74±1,91, and for the technique baro-thermo polymerization with an average of 24,92±1,83. For the technique of hot polymerization batch E, who has a preparation of 2 mm in the corono-radicular axe, recording an average of the extraction force of the endocrown of 23.56±2.05. Conclusion: The success of endodontic treatment also depends on prosthetic restoration for a long-term oral rehabilitation

Hybrid Surgery for symptomatic chronic internal carotid artery occlusion: a single-center experience.

Patients with symptomatic chronic internal carotid artery occlusion (ICAO) face a high risk of recurrent stroke despite receiving aggressive medical therapy. This study aimed to evaluate the effectiveness and safety of hybrid surgery in treating symptomatic chronic ICAO. This retrospective case series was conducted at a single center. From January 2019 to December 2022, patients with symptomatic chronic ICAO who underwent hybrid surgery were included. We collected baseline data, lesion characteristics, revascularization rates, perioperative complications, and follow-up outcomes. The study enrolled 27 patients, comprising 22 males and 5 females, with symptomatic chronic ICAO. The hybrid surgery achieved a technical success rate of 100% for revascularization (n = 27), with a perioperative complication rate of 14.8% (n = 4). Following a median follow-up of 6.0months (IQR, 4-10), 21 patients underwent a DSA or CT angiography reexamination, confirming a vascular patency rate of 90.5% (n = 19). One patient required surgery for severe in-stent restenosis, and another experienced asymptomatic occlusion. Clinical follow-ups were conducted for all 26 patients; no new strokes were reported in the qualifying artery territory, with 13 patients scoring 0, 12 scoring 1, and 1 scoring 2 on the mRS. Although hybrid surgery represent a promising option for treating chronic ICAO, they are also associated with a relatively high incidence of treatment-related complications. The application of composite surgery should be based on standardized technical guidelines and the careful selection of patients who are genuinely at high risk for recurrent strokes.

Maskrenderer: 3D-infused multi-mask realistic face reenactment

We present a novel end-to-end identity-agnostic face reenactment system, MaskRenderer, that can generate realistic, high fidelity frames in real-time. Although recent face reenactment works have shown promising results, there are still significant challenges such as identity leakage and imitating mouth movements, especially for large pose changes and occluded faces. MaskRenderer tackles these problems by using (i) a 3DMM to model 3D face structure to better handle pose changes, occlusion, and mouth movements compared to 2D representations; (ii) a triplet loss function to embed the cross-reenactment during training for better identity preservation; and (iii) multi-scale occlusion, improving inpainting and restoring missing areas. Comprehensive quantitative and qualitative experiments conducted on the VoxCeleb1 test set, demonstrate that MaskRenderer outperforms state-of-the-art models on unseen faces, especially when the Source and Driving identities are very different.

YOLO-FaceV2: A scale and occlusion aware face detector

In recent years, face detection algorithms based on deep learning have made great progress. Nevertheless, the effective utilization of face detectors for small and occlusion faces remains challenging, primarily stemming from the limitations in pixel information and the presence of missing features. In this paper, we propose a novel real-time face detector, YOLO-FaceV2, built upon the YOLOv5 architecture. Our approach introduces a Receptive Field Enhancement (RFE) module designed to extract multi-scale pixel information and augment the receptive field for accurately detecting small faces. To address issues related to face occlusion, we introduce an attention mechanism termed the Separated and Enhancement Attention Module (SEAM), which effectively focuses on the regions affected by occlusion. Furthermore, we propose a Slide Weight Function (SWF) to mitigate the imbalance between easy and hard samples. The experiments demonstrate that our YOLO-FaceV2 achieves performance exceeding the state-of-the-art on the WiderFace validation dataset. Source code and pre-trained model are available at https://github.com/Krasjet-Yu/YOLO-FaceV2.

Occlusion-aware deep convolutional neural network via homogeneous Tanh-transforms for face parsing

Face parsing infers a pixel-wise label map for each semantic facial component. Previous methods generally work well for uncovered faces, however, they overlook facial occlusion and ignore some contextual areas outside a single face, especially when facial occlusion has become a common situation during the COVID-19 epidemic. Inspired by the lighting phenomena in everyday life, where illumination from four distinct lamps provides a more uniform distribution than a single central light source, we propose a novel homogeneous tanh-transform for image preprocessing, which is made up of four tanh-transforms. These transforms fuse the central vision and the peripheral vision together. Our proposed method addresses the dilemma of face parsing under occlusion and compresses more information from the surrounding context. Based on homogeneous tanh-transforms, we propose an occlusion-aware convolutional neural network for occluded face parsing. It combines information in both Tanh-polar space and Tanh-Cartesian space, capable of enhancing receptive fields. Furthermore, we introduce an occlusion-aware loss to focus on the boundaries of occluded regions. The network is simple, flexible, and can be trained end-to-end. To facilitate future research of occluded face parsing, we also contribute a new cleaned face parsing dataset. This dataset is manually purified from several academic or industrial datasets, including CelebAMask-HQ, Short-video Face Parsing, and the Helen dataset, and will be made public. Experiments demonstrate that our method surpasses state-of-the-art methods in face parsing under occlusion.

Accelerated Stochastic Variance Reduction Gradient Algorithms for Robust Subspace Clustering.

Robust face clustering enjoys a wide range of applications for gate passes, surveillance systems and security analysis in embedded sensors. Nevertheless, existing algorithms have limitations in finding accurate clusters when data contain noise (e.g., occluded face clustering and recognition). It is known that in subspace clustering, the ℓ1- and ℓ2-norm regularizers can improve subspace preservation and connectivity, respectively, and the elastic net regularizer (i.e., the mixture of the ℓ1- and ℓ2-norms) provides a balance between the two properties. However, existing deterministic methods have high per iteration computational complexities, making them inapplicable to large-scale problems. To address this issue, this paper proposes the first accelerated stochastic variance reduction gradient (RASVRG) algorithm for robust subspace clustering. We also introduce a new momentum acceleration technique for the RASVRG algorithm. As a result of the involvement of this momentum, the RASVRG algorithm achieves both the best oracle complexity and the fastest convergence rate, and it reaches higher efficiency in practice for both strongly convex and not strongly convex models. Various experimental results show that the RASVRG algorithm outperformed existing state-of-the-art methods with elastic net and ℓ1-norm regularizers in terms of accuracy in most cases. As demonstrated on real-world face datasets with different manually added levels of pixel corruption and occlusion situations, the RASVRG algorithm achieved much better performance in terms of accuracy and robustness.

Improving masked communication: the case for transparent masks

Clear and compassionate communication is tantamount to quality of care and patient wellbeing. Yet, much of communication in healthcare occurs with lower faces visually hidden behind opaque face masks. Visual occlusion of the face by masks impairs both verbal and nonverbal communication, including recognition of basic emotions, perceptions of trustworthiness, emotional understanding, and empathic responding. This piece presents a brief overview of the new COVID-19 inspired literature which shows that visual face occlusion creates significant communication barriers which can be reduced by wearing transparent instead of opaque masks.

Face recognition with occluded face using improve intersection over union of region proposal network on Mask region convolutional neural network

Face recognition entails detecting and identifying facial attributes. Mask region convolutional neural network (R-CNN) method is a prominent approach, while prior research predominantly delved into refining loss functions and perfecting object and face detection, recognizing, and identifying faces using imperfect data remained relatively unexplored. This study focuses on an occluded dataset comprising Indonesian faces, wherein 'occluded' denotes facial data that lacks complete visibility-encompassing instances where objects obscure faces or are partially cropped. This investigation involves a deliberate experiment that tailors the intersection over union (IoU) of the region proposal network (RPN) to suit the nuances of occluded Indonesian faces, thereby augmenting accuracy in recognition and segmentation tasks. The innovation IoU in the strategic utilization of Anchors, which involves the exclusion of anchors falling beyond the image borders to optimize computational efficiency. The outcomes of this research are striking; it showcases a remarkable 14.75%, 10.9%, and 12.97% surge based on mean average precision (mAP), mean average recall (mAR), and F1-Scores compared to the conventional Mask R-CNN approach. Notably, our proposed model elevates the average accuracy by 10% to 15% and decreases running time by 21%, a noteworthy enhancement compared to the preceding model. This progress is substantiated by validation utilizing 300 instances dataset, reinforcing the robustness of our approach.

Generation of Face Privacy-Protected Images Based on the Diffusion Model.

In light of growing concerns about the misuse of personal data resulting from the widespread use of artificial intelligence technology, it is necessary to implement robust privacy-protection methods. However, existing methods for protecting facial privacy suffer from issues such as poor visual quality, distortion and limited reusability. To tackle this challenge, we propose a novel approach called Diffusion Models for Face Privacy Protection (DIFP). Our method utilizes a face generator that is conditionally controlled and reality-guided to produce high-resolution encrypted faces that are photorealistic while preserving the naturalness and recoverability of the original facial information. We employ a two-stage training strategy to generate protected faces with guidance on identity and style, followed by an iterative technique for improving latent variables to enhance realism. Additionally, we introduce diffusion model denoising for identity recovery, which facilitates the removal of encryption and restoration of the original face when required. Experimental results demonstrate the effectiveness of our method in qualitative privacy protection, achieving high success rates in evading face-recognition tools and enabling near-perfect restoration of occluded faces.

Human Emotion Detection Using CNN and Transfer Learning

Facial emotion detection is a critical component in human-computer interaction, mental health assessment, and security systems. In this project, we propose a robust facial emotion detection system leveraging state-of-the-art deep learning techniques. Our system utilizes Convolutional Neural Networks (CNNs) to extract meaningful features from facial images and classify them into five distinct emotional categories: neutral, surprise, sad, happy, and angry. We conducted extensive experiments on a diverse dataset consisting of over 10,000 annotated facial images collected from various sources. Through data augmentation techniques such as rotation, translation, and flipping, we expanded the dataset to enhance model training. Additionally, we employed transfer learning by fine-tuning a pre-trained CNN model, ResNet50, on our dataset to leverage its learned features. This project presents a system for real-time emotion monitoring using computer vision techniques. The system utilizes the Haar Cascade Classifier for face detection in live webcam video streams. Furthermore, we evaluated the system's performance across different lighting conditions, poses, and facial occlusions to assess its robustness in real-world scenarios. Our results indicate that the system maintains consistent performance across diverse conditions, making it suitable for deployment in applications requiring real-time facial emotion recognition.