Occluded Face Images Research Articles

A novel occluded face detection approach using Enhanced ORB and optimized GAN

Objectives:This research presents an approach to detect occluded face images. In order to achieve this, the research presents a novel technique that involves feature extraction and occluded face recognition. Methods: Feature extraction is performed by the enhanced ORB algorithm, which is proposed by the modification of the Oriented Fast and Rotated Brief (ORB) algorithm, by adding a phase for contrast adjustment, together with CNN features. For occluded face recognition, a Generative Adversarial Network (GAN) optimized by the proposed SR-SSA is designed. SR-SSA is proposed by the integration of Search and Rescue Optimization (SAR) in the Sparrow Search Algorithm (SSA). Results: The experimental results demonstrate that the SR-SSA-based GAN algorithm outperforms existing methods in terms of accuracy of 0.956, FAR of 0.045 and FRR of 0.021.

Adversarially Learning Occlusions by Backpropagation for Face Recognition.

With the accomplishment of deep neural networks, face recognition methods have achieved great success in research and are now being applied at a human level. However, existing face recognition models fail to achieve state-of-the-art performance in recognizing occluded face images, which are common scenarios captured in the real world. One of the potential reasons for this is the lack of large-scale training datasets, requiring labour-intensive and costly labelling of the occlusions. To resolve these issues, we propose an Adversarially Learning Occlusions by Backpropagation (ALOB) model, a simple yet powerful double-network framework used to mitigate manual labelling by contrastively learning the corrupted features against personal identity labels, thereby maximizing the loss. To investigate the performance of the proposed method, we compared our model to the existing state-of-the-art methods, which function under the supervision of occlusion learning, in various experiments. Extensive experimentation on LFW, AR, MFR2, and other synthetic masked or occluded datasets confirmed the effectiveness of the proposed model in occluded face recognition by sustaining better results in terms of masked face recognition and general face recognition. For the AR datasets, the ALOB model outperformed other advanced methods by obtaining a 100% recognition rate for images with sunglasses (protocols 1 and 2). We also achieved the highest accuracies of 94.87%, 92.05%, 78.93%, and 71.57% TAR@FAR = 1 × 10-3 in LFW-OCC-2.0 and LFW-OCC-3.0, respectively. Furthermore, the proposed method generalizes well in terms of FR and MFR, yielding superior results in three datasets, LFW, LFW-Masked, and MFR2, and producing accuracies of 98.77%, 97.62%, and 93.76%, respectively.

Knowledge Distillation Hashing for Occluded Face Retrieval

Deep hashing has proven to be efficient and effective for large-scale face retrieval. However, existing hashing methods are designed for normal face images only. They fail to consider the fact that face images may be occluded because of wearing masks, hats, glasses, etc. Retrieval performance of existing face retrieval methods is much worse when dealing with occluded face images. In this work, we propose the knowledge distillation hashing (KDH) to deal with occluded face images. The KDH is a two-stage learning approach with teacher-student model distillation. We first train a teacher hashing network using normal face images and then the knowledge from teacher model is used to guide the optimization of the student model using occluded face images as input only. With knowledge distillation, we build a connection between imperfect face information and the optimal hash codes. Experimental results show that the KDH yields significant improvements and better retrieval performance in comparison to existing state-of-the-art deep hashing retrieval methods under six different face occlusion situations.

Two-Stream Prototype Learning Network for Few-Shot Face Recognition Under Occlusions

Few-shot face recognition under occlusion (FSFRO) aims to recognize novel subjects given only a few, probably occluded face images, and it is challenging and common in real-world scenarios. Unknown occlusions may deteriorate the class prototypes, while an occluded image in the support set may be critical for recognition if the query image is occluded. This motivates us to propose a novel Two-stream Prototype Learning Network (TSPLN) for FSFR under occlusions by simultaneously considering the quality of support images and their relevance to the query i mage. Specifically, we design a two-stream architecture, which mainly consists of a support-centered stream and query-centered stream, to learn the optimal class prototypes. The former stream is to reduce the negative impact of occluded images on the prototype. This is achieved by exploring the similarities between different images in the support set. In the query-centered stream, we exploit the relevance between the query and support set based on feature alignment (FA). We conduct extensive experiments on two popular datasets: CASIA-WebFace and RMFRD. The experimental results show that our proposed method achieves the state-of-the-art performance for occluded face recognition in the few-shot setting.

Generative face inpainting hashing for occluded face retrieval.

COVID-19 has resulted in a significant impact on individual lives, bringing a unique challenge for face retrieval under occlusion. In this paper, an occluded face retrieval method which consists of generator, discriminator, and deep hashing retrieval network is proposed for face retrieval in a large-scale face image dataset under variety of occlusion situations. In the proposed method, occluded face images are firstly reconstructed using a face inpainting model, in which the adversarial loss, reconstruction loss and hash bits loss are combined for training. With the trained model, hash codes of real face images and corresponding reconstructed face images are aimed to be as similar as possible. Then, a deep hashing retrieval network is used to generate compact similarity-preserving hashing codes using reconstructed face images for a better retrieval performance. Experimental results show that the proposed method can successfully generate the reconstructed face images under occlusion. Meanwhile, the proposed deep hashing retrieval network achieves better retrieval performance for occluded face retrieval than existing state-of-the-art deep hashing retrieval methods.

End2End Occluded Face Recognition by Masking Corrupted Features.

With the recent advancement of deep convolutional neural networks, significant progress has been made in general face recognition. However, the state-of-the-art general face recognition models do not generalize well to occluded face images, which are exactly the common cases in real-world scenarios. The potential reasons are the absences of large-scale occluded face data for training and specific designs for tackling corrupted features brought by occlusions. This article presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks. In addition, we construct massive occluded face images to train FROM effectively and efficiently. FROM is simple yet powerful compared to the existing methods that either rely on external detectors to discover the occlusions or employ shallow models which are less discriminative. Experimental results on the LFW, Megaface challenge 1, RMF2, AR dataset and other simulated occluded/masked datasets confirm that FROM dramatically improves the accuracy under occlusions, and generalizes well on general face recognition.

A New approach to Recognize Human Face Under Unconstrained Environment

Human face is considered as one of the most useful traits in biometrics, and it has been widely used in education, security, military and many other applications. However, in most of currently deployed face recognition systems ideal imaging conditions are assumed; to capture a fully featured images with enough quality to perform the recognition process. As the unmasked face will have a considerable impact on the numbers of new infections in the era of COVID-19 pandemic, a new unconstrained partial facial recognition method must be developed. In this research we proposed a mask detection method based on HOG (Histogram of Gradient) features descriptor and SVM (Support Vector Machine) to determine whether the face is masked or not, the proposed method was tested over 10000 randomly selected images from Masked Face-Net database and was able to correctly classify 98.73% of the tested images. Moreover, and to extract enough features from partially occluded face images, a new geometrical features extraction algorithm based on Contourlet transform was proposed. The method achieved 97.86% recognition accuracy when tested over 4784 correctly masked face images from Masked Face-Net database. Keywords: Facial Recognition, Unconstraint conditions, masked faces, HOG, Support Vector Machine.

Face occlusion removal for face recognition using the related face by structural similarity index measure and principal component analysis

Facial occlusions like sunglasses, masks, caps etc. have severe consequences when reconstructing the partially occluded regions of a facial image. This paper proposes a novel hybrid machine learning approach for occlusion removal based on Structural Similarity Index Measure (SSIM) and Principal Component Analysis (PCA), called SSIM_PCA. The proposed system comprises two stages. In the first stage, a Face Similar Matrix (FSM) guided by the Structural Similarity Index Measure is generated to provide the necessary information to recover from the lost regions of the face image. The FSM generates Related Face (RF) images similar to the probe image. In the second stage, these RF images are considered as related information and used as input data to generate eigenspaces using PCA to reconstruct the occluded face region exploiting the relationship between the occluded region and related face images, which contain relevant data to recover from the occluded area. Experimental results with five standard datasets viz. Caspeal-R1, IMFDB, and FEI have proven that the proposed method works well under illumination changes and occlusion of facial images.

Face Dataset Augmentation with Generative Adversarial Network

Face recognition is a widely used scene of artificial intelligence technology. However, some face occlusions cause the face to be unable to be effectively detected in a specific environment. Although many algorithms have been proposed to solve this problem, in essence, a large number of face image data containing occlusion elements is needed to train to improve the detection ability of the algorithm. In recent years, this problem can be effectively solved by using the image generation ability of generative adversarial network. This paper proposes an improved Generative Adversarial Networks (GAN), which improves the effect of occluded face image generation by adding coding module. Through the expansion of data set, the detection accuracy of several classic face detection models for occluded faces is improved by more than 3%. At the moment when the epidemic has not been over, occlusion face data is of great significance to improve the performance of face detection systems in specific public places such as customs security inspection and medical centers.

Facial expression recognition based on deep learning

Background and objectiveFacial expression recognition technology will play an increasingly important role in our daily life. Autonomous driving, virtual reality and all kinds of robots integrated into our life depend on the development of facial expression recognition technology. Many tasks in the field of computer vision are based on deep learning technology and convolutional neural network. The paper proposes an occluded expression recognition model based on the generated countermeasure network. The model is divided into two modules, namely, occluded face image restoration and face recognition. MethodsFirstly, this paper summarizes the research status of deep facial expression recognition methods in recent ten years and the development of related facial expression database. Then, the current facial expression recognition methods based on deep learning are divided into two categories: Static facial expression recognition and dynamic facial expression recognition. The two methodswill be introduced and summarized respectively. Aiming at the advanced deep expression recognition algorithms in the field, the performance of these algorithms on common expression databases is compared, and the strengths and weaknesses of these algorithms are analyzed in detail. Discussion and resultsAs the task of facial expression recognition is gradually transferred from the controlled laboratory environment to the challenging real-world environment, with the rapid development of deep learning technology, deep neural network can learn discriminative features, and is gradually applied to automatic facial expression recognition task. The current deep facial expression recognition system is committed to solve the following two problems: (1) Overfitting due to lack of sufficient training data; (2) In the real world environment, other variables that have nothing to do with expression bring interference problems. ConclusionFrom the perspective of algorithm, combining other expression models, such as facial action unit model and pleasure arousal dimension model, as well as other multimodal models, such as audio mode, 3D face depth information and human physiological information, can make expression recognition more practical.

A Robust Face Recognition Algorithm Based on an Improved Generative Confrontation Network

Objective: In practical applications, an image of a face is often partially occluded, which decreases the recognition rate and the robustness. Therefore, in response to this situation, an effective face recognition model based on an improved generative adversarial network (GAN) is proposed. Methods: First, we use a generator composed of an autoencoder and the adversarial learning of two discriminators (local discriminator and global discriminator) to fill and repair an occluded face image. On this basis, the Resnet-50 network is used to perform image restoration on the face. In our recognition framework, we introduce a classification loss function that can quantify the distance between classes. The image generated by the generator can only capture the rough shape of the missing facial components or generate the wrong pixels. To obtain a clearer and more realistic image, this paper uses two discriminators (local discriminator and global discriminator, as mentioned above). The images generated by the proposed method are coherent and minimally influence facial expression recognition. Through experiments, facial images with different occlusion conditions are compared before and after the facial expressions are filled, and the recognition rates of different algorithms are compared. Results: The images generated by the method in this paper are truly coherent and have little impact on facial expression recognition. When the occlusion area is less than 50%, the overall recognition rate of the model is above 80%, which is close to the recognition rate pertaining to the non-occluded images. Conclusions: The experimental results show that the method in this paper has a better restoration effect and higher recognition rate for face images of different occlusion types and regions. Furthermore, it can be used for face recognition in a daily occlusion environment, and achieve a better recognition effect.

Assessing the Performance of DWT-PCA/SVD Face Recognition Algorithm under Multiple Constraints

Many architectures of face recognition modules have been developed to tackle the challenges posed by varying environmental constraints such as illumination, occlusions, pose, and expressions. These recognition systems have mainly focused on a single constraint at a time and have achieved remarkable successes. However, the presence of multiple constraints may deteriorate the performance of these face recognition systems. In this study, we assessed the performance of Principal Component Analysis and Singular Value Decomposition using Discrete Wavelet Transform (DWT-PCA/SVD) for preprocessing face recognition algorithm on multiple constraints (partially occluded face images acquired with varying expressions). Numerical evaluation of the study algorithm gave reasonably average recognition rates of 77.31% and 76.85% for left and right reconstructed face images with varying expressions, respectively. A statistically significant difference was established between the average recognition distance of the left and right reconstructed face images acquired with varying expressions using pairwise comparison test. The post hoc analysis using the Bonferroni simultaneous confidence interval revealed that the significant difference established through the pairwise comparison test was mainly due to the sad expressions. Although the performance of the DWT-PCA/SVD algorithm declined as compared to its performance on single constraints, the algorithm attained appreciable performance level under multiple constraints. The DWT-PCA/SVD recognition algorithm performs reasonably well for recognition when partial occlusion with varying expressions is the underlying constraint.

Assessing the Effect of Data Augmentation on Occluded Frontal Faces Using DWT-PCA/SVD Recognition Algorithm

The drift towards face-based recognition systems can be attributed to recent advances in supportive technology and emerging areas of application including voting systems, access control, human-computer interactions, entertainments, and crime control. Despite the obvious advantages of such systems being less intrusive and requiring minimal cooperation of subjects, the performances of their underlying recognition algorithms are challenged by the quality of face images, usually acquired from uncontrolled environments with poor illuminations, varying head poses, ageing, facial expressions, and occlusions. Although several researchers have leveraged on the property of bilateral symmetry to reconstruct half-occluded face images, their approach becomes deficient in the presence of random occlusions. In this paper, we harnessed the benefits of the multiple imputation by the chained equation technique and image denoising using Discrete Wavelet Transforms (DWTs) to reconstruct degraded face images with random missing pixels. Numerical evaluation of the study algorithm gave a perfect (100%) average recognition rate each for recognition of occluded and augmented face images. The study also revealed that the average recognition rate for the augmented face images (75.5811) was significantly lower than the average recognition rate (430.7153) of the occluded face images. MICE augmentation is recommended as a suitable data enhancement mechanism for imputing missing data/pixel of occluded face images.

Recognition of Augmented Frontal Face Images Using FFT-PCA/SVD Algorithm

In spite of the differences in visual stimulus of human beings such as ageing, changing conditions of a person, and occlusion, recognition can even be done at a glance by the human eye many years after the previous encounter. It has been established that facial differences like the hairstyle changes, growing of one’s beard, wearing of glasses, and other forms of occlusions can hardly hinder the power of the human brain from making a face recognition. However, the same cannot easily be said about automated intelligent systems which have been developed to mimic the skill of the human brain to aid in recognition. There have been growing interests in developing a resilient and efficient recognition system mainly because of its numerous application areas (access control, entertainment/leisure, security system based on biometric data, and user-friendly human-machine interfaces). Although there have been numerous researches on face recognition under varying pose, illumination, expression, and image degradations, problems caused by occlusions are mostly ignored. This study thus focuses on facial occlusions and proposes an enhancement mechanism through face image augmentation to improve the recognition of occluded face images. This study assessed the performance of Principal Component Analysis with Singular Value Decomposition using Fast Fourier Transform (FFT-PCA/SVD) for preprocessing face recognition algorithm on face images with missingness and augmented face image database. It was found that the average recognition rates for the FFT-PCA/SVD algorithm were the same ( 90 % ) when face images with missingness and augmented face images were used as test images, respectively. The statistical evaluation revealed that there exists a significant difference in the average recognition distances for the face images with missingness and augmented face images when FFT-PCA/SVD is used for recognition. Augmented face images tend to have a relatively lower average recognition distance when used as test images. This finding is contrary to the equal performance assessment by the adopted numerical technique. The MICE algorithm is therefore recommended as a suitable imputation mechanism for enhancing/improving the performance of the face recognition system.

Sparse representation for face recognition: A review paper

With the increasing use of surveillance cameras, face recognition is being studied by many researchers for security purposes. Although high accuracy has been achieved for frontal faces, the existing methods have shown poor performance for occluded and corrupt images. Recently, sparse representation based classification (SRC) has shown the state-of-the-art result in face recognition on corrupt and occluded face images. Several researchers have developed extended SRC methods in the last decade. This paper mainly focuses on SRC and its extended methods of face recognition. SRC methods have been compared on the basis of five issues of face recognition such as linear variation, non-linear variation, undersampled, pose variation, and low resolution. Detailed analysis of SRC methods for issues of face recognition have been discussed based on experimental results and execution time. Finally, the limitation of SRC methods have been listed to help the researchers to extend the work of existing methods to resolve the unsolved issues.

Recognition algorithm of athletes' partially occluded face based on a deep learning algorithm

Because there are a lot of noise data in the partially occluded face image, the existing recognition methods have the problems of low recall rate and long time consumption. In this paper, a new recognition algorithm based on a deep learning algorithm is proposed. This method uses boosting algorithm to locate face information, based on which the face image is greyed and denoised. The local binary pattern is used to extract face features, and the convolution neural network in deep learning algorithm is used to realise face feature recognition. The experimental results show that compared with the traditional face feature recognition algorithm, the proposed method has significantly improved recognition accuracy and recall rate, and the feature recognition time is shorter, which proves that the proposed algorithm has stronger application performance.

A robust two quadrant sparse classifier for partially occluded face image recognition

In the last decade, Sparse representation method of classification has proved its efficiency for face recognition applications. The major challenge for face recognition algorithms is partial occlusion and expression changes in face images. In case of upper or lower face occlusion, a large portion of discriminative information gets missing which results into lower classification accuracy. In this paper a novel two quadrant sparse classifier is proposed which implements sparse representation in two different quadrants. Proposed classifier uses visual face information present in upper and lower quadrants independently to classify the test face image which provides correct classification even if one quadrant is occluded. The enhanced accuracy of proposed technique is proved using extensive simulations carried out on two standard databases (ORL and YALE). The experimental results for all number of training images and training sets are compared with simple sparse method in terms of mean classification accuracy. The performance of proposed technique is analysed with a maximum improvement of 3.51 % in terms of classification accuracy.

Robust face alignment by cascaded regression and de-occlusion

Face alignment is a typical facial behavior analysis task in computer vision. However, the performance of face alignment is degraded greatly when the face image is partially occluded. In order to achieve better mapping between facial appearance features and shape increments, we propose a robust and occlusion-free face alignment algorithm in which a face de-occlusion module and a deep regression module are integrated into a cascaded deep generative regression model. The face de-occlusion module is a disentangled representation learning Generative Adversarial Networks (GANs) which aims to locate occlusions and recover the genuine appearance from partially occluded face image. The deep regression module can enhance facial appearance representation by utilizing the recovered faces to obtain more accurate regressors. Then, by the cascaded deep generative regression model, we recover the partially occluded face image and achieve accurate locating of landmarks gradually. It is interesting to show that the cascaded deep generative regression model can effectively locate occlusions and recover more genuine faces, which can be further used to improve the performance of face alignment. Experimental results conducted on four challenging occluded face datasets demonstrate that our method outperforms state-of-the-art methods.

Occlusion Images Generation from Occlusion-Free Images for Criminals Identification based on Artificial Intelligence Using Image

In this paper, we propose the method to generate occlusion images which include sunglass and scarf from occlusion-free images for criminals identification. First, we construct an auxiliary set that has occlusion-free face image and its occluded face images from AR database. Secondly, pixel weights are determined by the statistical analysis of the degree depending on whether occlusion-free or occlusion. As the weight is closer to 1, the pixel value of original image is maintained as much as possible. Therefore, pixel values of occlusion-free region are preserved when the weights are closed to 1. Third, pixel values of the occlusion region of original image are replaced to pixel values which are the mean of occlusion images by multiplying the composited weights. As a result of generate face images with various occlusions using the proposed method, the images were able to preserve the person-specific characteristic, and criminals identification has become easier.

The Application of Modified Least Trimmed Squares with Genetic Algorithms Method in Face Recognition

Severely occluded face images are the main problem in low performance of face recognition algorithms. In this paper, we apply a new algorithm, a modified version of the least trimmed squares (LTS) with a genetic algorithms introduce by <!--[if supportFields]><i><span
 style='font-size:9.0pt;font-family:"Arial","sans-serif";mso-fareast-font-family:
 "Times New Roman";color:black;mso-ansi-language:EN-US;mso-fareast-language:
 EN-US;mso-bidi-language:AR-SA'><span style='mso-element:field-begin'></span><span
 style='mso-spacerun:yes'> </span>ADDIN EN.CITE
 <EndNote><Cite><Author>Rahim
 Abdul</Author><Year>2016</Year><RecNum>303</RecNum><DisplayText>[1]</DisplayText><record><rec-number>303</rec-number><foreign-keys><key
 app="EN" db-id="f5s5xeavm025z9expeavfwr2x2szra2tx55s"
 timestamp="1507074147">303</key></foreign-keys><ref-type
 name="Journal
 Article">17</ref-type><contributors><authors><author>Rahim
 Abdul, Nur Azimah</author><author>Ramli, Norazan
 Mohamed</author><author>Md Ghani, Nor Azura</author></authors></contributors><titles><title>The
 Performance of Modified Least Trimmed Squares-Based Methods for Large Data Sets
 Based on Monte Carlo Simulations</title><secondary-title>Advanced
 Science Letters</secondary-title></titles><periodical><full-title>Advanced
 Science
 Letters</full-title></periodical><pages>4359-4363</pages><volume>22</volume><number>12</number><keywords><keyword>Genetic
 Algorithm</keyword><keyword>Large Data</keyword><keyword>Least
 Trimmed Squares</keyword></keywords><dates><year>2016</year><pub-dates><date>//</date></pub-dates></dates><urls><related-urls><url>http://www.ingentaconnect.com/content/asp/asl/2016/00000022/00000012/art00091</url><url>https://doi.org/10.1166/asl.2016.8155</url></related-urls></urls><electronic-resource-num>10.1166/asl.2016.8155</electronic-resource-num></record></Cite></EndNote><span
 style='mso-element:field-separator'></span></span></i><![endif]-->[1]<!--[if supportFields]><i><span
 style='font-size:9.0pt;font-family:"Arial","sans-serif";mso-fareast-font-family:
 "Times New Roman";color:black;mso-ansi-language:EN-US;mso-fareast-language:
 EN-US;mso-bidi-language:AR-SA'><span style='mso-element:field-end'></span></span></i><![endif]-->. We focused on the application of modified LTS with genetic algorithm method for face image recognition. This algorithm uses genetic algorithms to construct a basic subset rather than selecting the basic subset randomly. The modification in this method lessens the number of trials to obtain the minimum of the LTS objective function. This method was then applied to two benchmark datasets with clean and occluded query images. The performance of this method was measured by recognition rates. The AT&T dataset and Yale Dataset with different image pixel sizes were used to assess the method in performing face recognition. The query images were contaminated with salt and pepper noise. The modified LTS with GAs method is applied in face recognition framework by using the contaminated images as query image in the context of linear regression. By the end of this study, we can determine this either this method can perform well in dealing with occluded images or vice versa.