YouTube Faces Research Articles

Enhanced Face Detection in Videos Based on Integrating Spatial Features (LBP, CS-LBP) with CNN Technique

Face detection is a crucial aspect of computer vision and image processing, in order to enable the automatic detection and identification of human faces in video streams, face detection is an essential component of computer vision and image processing. Applications for facial recognition, video analytics, security systems, and surveillance all depend on it. Face identification techniques face many obstacles and issues, such as positional fluctuations, illumination changes, resolution and scale issues, facial emotions, and cosmetics. Robust algorithms are required for efficient face detection. This field looks at the feature extraction process using a variety of techniques. These consist of the center symmetric local binary patterns (CS_LBP) approach and the local binary patterns (LBP) method. The YouTube Face database provided the video frames that we used for our study. In order to train the convolutional neural network (CNN) to detect human faces in the video and draw a bounding box around them. The experimental results of the suggested approaches show that. The accuracy rate was 94% higher with the LBP techniques. However, the CS_LBP technique showed the best level of accuracy in both face detection and face rectangle recognition, with an accuracy rate of 95%.

Privacy-preserving model for biometric-based authentication and Key Derivation Function

Bio-cryptosystems often save the biometric template for authentication and generally employ randomly generated keys to encrypt and sign data. This method raises privacy protection concerns. Furthermore, for a system secured by the usage of a cryptographic key, losing the key often has disastrous consequences. To overcome the privacy issues, and allow the secure recovery of lost keys, we design a Key Derivation Function to extract a key from biometric data: a new method – based on clustering algorithms – detects consistent and discriminative features from biometric characteristics to create a code. Then, HMAC-SHA256 (as specified by the National Institute of Standards and Technology) generates a standard key from the code. To reproduce the code at future times (that also serves for authentication), the Key Derivation Function stores helper data with the guarantee of privacy. Indeed, with a private face dataset, the probability of generating the code with only the helper data is less than 2−300, and less than 2−246 for a subset of the YouTube Face database. Moreover, on the private database and the tested users from the YouTube Face database, our system has a false acceptance rate of 0%. It corrects up to 40.3% of noise levels on the private database and has good management of the inter-user variability.

Neural dynamics for improving optimiser in deep learning with noise considered

AbstractAs deep learning evolves, neural network structures become increasingly sophisticated, bringing a series of new optimisation challenges. For example, deep neural networks (DNNs) are vulnerable to a variety of attacks. Training neural networks under privacy constraints is a method to alleviate privacy leakage, and one way to do this is to add noise to the gradient. However, the existing optimisers suffer from weak convergence in the presence of increased noise during training, which leads to a low robustness of the optimiser. To stabilise and improve the convergence of DNNs, the authors propose a neural dynamics (ND) optimiser, which is inspired by the zeroing neural dynamics originated from zeroing neural networks. The authors first analyse the relationship between DNNs and control systems. Then, the authors construct the ND optimiser to update network parameters. Moreover, the proposed ND optimiser alleviates the non‐convergence problem that may be suffered by adding noise to the gradient from different scenarios. Furthermore, experiments are conducted on different neural network structures, including ResNet18, ResNet34, Inception‐v3, MobileNet, and long and short‐term memory network. Comparative results using CIFAR, YouTube Faces, and R8 datasets demonstrate that the ND optimiser improves the accuracy and stability of DNNs under noise‐free and noise‐polluted conditions. The source code is publicly available at https://github.com/LongJin‐lab/ND.

ALMEGA-VIR: face video retrieval system

ABSTRACT The limitations of Content Based Video Retrieval (CBVR), such as large pools of video data, selection of features, limited processing capacity, and content-related issues, can be overcome by Deep Belief Neural Networks (DBNs). The search engine does the processing, and the results are effectively returned to the users. The deep learning model also has some comparable challenges to be solved at various levels. Typically, the selection of frames is one of the most essential tasks for a face retrieval system which is efficiently accomplished by using deep learning models. Also, the performance of recognition models highly depends on the selection of frames that include both low-level and high-level features from the given video sequence. This paper proposes ALPHA-TO-OMEGA (ALMEGA-VIR) Visual Information Retrieval scheme that analyzes the contents of the video file for a particular face and retrieves the user's relevant videos. VIR aims to solve the challenge of locating relevant pictures and videos based on a query. VIR can be based on text-based search, content-based search, and Mixed based search. VIR here focuses on content-based search approach. The YouTube Celebrities (YTC) dataset, the Youtube Face (YTF) dataset, and Honda / UCSD dataset are used in this work. The used dataset addresses issues such as illumination changes, occlusion, rotation, and scale. We propose an ALMEGA-VIR framework for visual novelty face identification for specific investigation and evaluation. The classification accuracy of the proposed ALMEGA-VIR model shows that 4% is more effective than traditional classifier algorithms such as SVM, Naïve Bayesian classifier, MLP and Random Forest classifier. A comparative study is also implemented with other existing works to check the superiority of the proposed method.

Truth-value unconstrained face clustering for identity resolution in a distributed environment of criminal police information systems

The work of criminal police in contemporary society is characterized by the proliferation of data and information to be processed, a more significant limitation of access to personal data, increased public monitoring, and higher expectations in the efficiency of identifying perpetrators. Also, all information collected during investigations is distributed through police systems separated by different levels and organization units, and often there are insufficient human and material resources. Resolving the perpetrator’s identity in those circumstances is a complex task, and police decision support systems must group all available evidence related to specific persons. For that purpose, this paper proposes a new approach to unconstrained pairwise clustering of face feature vectors extracted from the histogram of oriented gradients descriptor, named Truth-value clustering (TVC), based on non-axiomatic logic and graphs. The clustering approach was experimentally tested with six different face image databases. They were created to simulate unconstrained conditions like IARPA Janus Benchmark-B Face Dataset (IJB-B), IMDb-Wiki, Labeled Faces in the Wild, and YouTube Faces. The results of the proposed solution are compared with other state-of-the-art methods, showing that the approach gives, in summary, significantly better results. Application of the IJB-B protocol created for testing face clustering showed that the new approach gives better results by an average of 8.25% (σ=4.2). The main advantage over other methods is the possibility of utilizing mechanisms from non-axiomatic logic such as revision, which can then acquire new knowledge based on information from different nodes of the distributed environment consisting of various police information systems.

Attendance Compilation by Facial Recognition Methods of Image Processing: A Review

Abstract: In recent years, researchers have continued to refine and improve deep learning-based approaches to image processing, as well as exploring new areas such as generative adversarial networks (GANs) and reinforcement learning. This paper provides a comprehensive survey of deep learning-based methods for face recognition, including CNN-based models, auto encoder models, and hybrid models. These papers demonstrate the effectiveness of deep convolutional neural networks (CNNs) in face detection, recognition, and attendance compilation, achieving state-of-the-art accuracy on several benchmark datasets, including LFW, YouTube Faces, and YTF datasets. The Efficient Net model is a family of CNNs that achieves state of the art accuracy on multiple image recognition benchmarks while being significantly smaller and faster than previous models. The Arc Face loss function is used for facial landmark detection and gender classification in facial images.

UFace: An Unsupervised Deep Learning Face Verification System

Deep convolutional neural networks are often used for image verification but require large amounts of labeled training data, which are not always available. To address this problem, an unsupervised deep learning face verification system, called UFace, is proposed here. It starts by selecting from large unlabeled data the k most similar and k most dissimilar images to a given face image and uses them for training. UFace is implemented using methods of the autoencoder and Siamese network; the latter is used in all comparisons as its performance is better. Unlike in typical deep neural network training, UFace computes the loss function k times for similar images and k times for dissimilar images for each input image. UFace’s performance is evaluated using four benchmark face verification datasets: Labeled Faces in the Wild (LFW), YouTube Faces (YTF), Cross-age LFW (CALFW) and Celebrities in Frontal Profile in the Wild (CFP-FP). UFace with the Siamese network achieved accuracies of 99.40%, 96.04%, 95.12% and 97.89%, respectively, on the four datasets. These results are comparable with the state-of-the-art methods, such as ArcFace, GroupFace and MegaFace. The biggest advantage of UFace is that it uses much less training data and does not require labeled data.

ApaNet: adversarial perturbations alleviation network for face verification.

Albeit Deep neural networks (DNNs) are widely used in computer vision, natural language processing and speech recognition, they have been discovered to be fragile to adversarial attacks. Specifically, in computer vision, an attacker can easily deceive DNNs by contaminating an input image with perturbations imperceptible to humans. As one of the important vision tasks, face verification is also subject to adversarial attack. Thus, in this paper, we focus on defending against the adversarial attack for face verification to mitigate the potential risk. We learn a network via an implementation of stacked residual blocks, namely adversarial perturbations alleviation network (ApaNet), to alleviate latent adversarial perturbations hidden in the input facial image. During the supervised learning of ApaNet, only the Labeled Faces in the Wild (LFW) is used as the training set, and the legitimate examples and corresponding adversarial examples produced by projected gradient descent algorithm compose supervision and inputs respectively. By leveraging the middle and high layer’s activation of FaceNet, the discrepancy between an image output by ApaNet and the supervision is calculated as the loss function to optimize ApaNet. Empirical experiment results on the LFW, YouTube Faces DB and CASIA-FaceV5 confirm the effectiveness of the proposed defender against some representative white-box and black-box adversarial attacks. Also, experimental results show the superiority performance of the ApaNet as comparing with several currently available techniques.

Face Verification Based on Deep Learning for Person Tracking in Hazardous Goods Factories

Person tracking in hazardous goods factories can provide a significant improvement in security and safety. This article proposes a face verification model which can be used to record travel paths for staff or related persons in the factory. As face images are captured from the dynamic crowd at entrance–exit gates of workshops, face verification is challenged by polymorphic faces, poor illumination and changing of a person’s pose. To adapt to this situation, a new face verification model is proposed, which is composed of two advanced deep learning neural network models. Firstly, MTCNN (Multi-Task Cascaded Convolutional Neural Network) is used to construct a face detector. Based on the SphereFace-20 network model, we have reconstructed a convolutional network architecture with the embedded Batch Normalization elements and the optimized network parameters. The new model, which is called the MDCNN, is used to extract efficient face features. A set of specific processing algorithms is used in the model to process polymorphic face images. The multi-view faces and various types of face images are used to train the models. The experimental results have demonstrated that the proposed model outperforms most existing methods on benchmark datasets such as the Labeled Faces in the Wild (LFW) and YouTube Face (YTF) datasets without multi-view (accuracy is 99.38% and 94.30%, respectively) and the CNBC/FERET datasets with multi-view (accuracy is 94.69%).

Knock Knock, Who’s There: Facial Recognition using CNN-based Classifiers

Artificial intelligence (AI) has captured the public’s imagination. Performance gains in computing hardware, and the ubiquity of data have enabled new innovations in the field. In 2014, Facebook’s DeepFace AI took the facial recognition industry by storm with its splendid performance on image recognition. While newer models exist, DeepFace was the first to achieve near-human level performance. To better understand how this breakthrough performance was achieved, we developed our own facial image detection models. In this paper, we developed and evaluated six Convolutional Neural Net (CNN) models inspired by the DeepFace architecture to explore facial feature identification. This research made use of the You Tube Faces (YTF) dataset which included 621,126 images consisting of 1,595 identities. Three models leveraged pretrained layers from VGG16 and InceptionResNetV2, whereas the other three did not. Our best model achieved a 84.6% accuracy on the test dataset.

Context-Aware Contribution Estimation for Feature Aggregation in Video Face Recognition

The difficulties in video-based face recognition, such as dramatic pose variations and low quality, can be alleviated by leveraging the rich complementary information between the frames. However, limited by the mini-batch training strategy, the current deep learning methods only utilizes the frames in each batch during training, which ignore the context of the entire video. In this paper, we propose a context-aware feature aggregation scheme to aggregate complementary information between different frames. Firstly, a two-branch structure is designed as the Context-aware feature Aggregation Network (CAN). Secondly, a context-aware training strategy using context bank is proposed, which alleviates the limitation of mini-batch samples by using the context of the entire video or several images belonging to the same ID and thus achieves global contribution estimation result. Comparative studies on benchmark datasets, such as IJB-C, YouTube Face (YTF), PaSC and COX, confirm that the proposed approach can achieve state-of-the-art level. Meanwhile, qualitative analysis on Multi-PIE dataset indicates that the contribution learned by the CAN is reasonable and beneficial to video face recognition.

MC-GCN: A Multi-Scale Contrastive Graph Convolutional Network for Unconstrained Face Recognition With Image Sets.

In this paper, a Multi-scale Contrastive Graph Convolutional Network (MC-GCN) method is proposed for unconstrained face recognition with image sets, which takes a set of media (orderless images and videos) as a face subject instead of single media (an image or video). Due to factors such as illumination, posture, media source, etc., there are huge intra-set variances in a face set, and the importance of different face prototypes varies considerably. How to model the attention mechanism according to the relationship between prototypes or images in a set is the main content of this paper. In this work, we formulate a framework based on graph convolutional network (GCN), which considers face prototypes as nodes to build relations. Specifically, we first present a multi-scale graph module to learn the relationship between prototypes at multiple scales. Moreover, a Contrastive Graph Convolutional (CGC) block is introduced to build attention control model, which focuses on those frames with similar prototypes (contrastive information) between pair of sets instead of simply evaluating the frame quality. The experiments on IJB-A, YouTube Face, and an animal face dataset clearly demonstrate that our proposed MC-GCN outperforms the state-of-the-art methods significantly.

Improved Face Recognition With Fractal-Based Texture Analysis

Fractals are useful to uniquely represent texture in the human face, which serves as an equivalent of human vision. FaceNet, calculating face descriptors of a person, has been observed to perform with setbacks when several factors of occlusion are present. This paper proposes a new methodology that exploits the self-similar patterns in a person's face to highlight and enhance regions of high texture in a facial image. The system maps the original image into a representation in the pre-processing stage of computer vision. This representation when fed as an input to the FaceNet CNN optimizes the face embedding generated. An SVM classifier separates the hard positive examples from the hard negative examples during classification. The model is trained using YouTube Faces DB as primary dataset and for validation; a custom dataset is designed to verify a person's identity despite the presence of secondary factors such as expressions and forgery. The proposed model attained an overall accuracy of 96.73% with the YouTube Faces DB, and a notable reduction in the false positive rates is observed.

Few-Shot Learning with a Novel Voronoi Tessellation-Based Image Augmentation Method for Facial Palsy Detection

Face palsy has adverse effects on the appearance of a person and has negative social and functional consequences on the patient. Deep learning methods can improve face palsy detection rate, but their efficiency is limited by insufficient data, class imbalance, and high misclassification rate. To alleviate the lack of data and improve the performance of deep learning models for palsy face detection, data augmentation methods can be used. In this paper, we propose a novel Voronoi decomposition-based random region erasing (VDRRE) image augmentation method consisting of partitioning images into randomly defined Voronoi cells as an alternative to rectangular based random erasing method. The proposed method augments the image dataset with new images, which are used to train the deep neural network. We achieved an accuracy of 99.34% using two-shot learning with VDRRE augmentation on palsy faces from Youtube Face Palsy (YFP) dataset, while normal faces are taken from Caltech Face Database. Our model shows an improvement over state-of-the-art methods in the detection of facial palsy from a small dataset of face images.

Ordered Weighted Aggregation Networks for Video Face Recognition

Video face recognition generally includes a step where all descriptors extracted for each frame are aggregated to generate a single video face representation. The most commonly used operator for aggregation is average, which gives the same relevance to each frame. Some adaptive aggregation algorithms have been developed, but most of them rely on the use of weighted mean as aggregation operator, thus disregarding many other types of aggregation operators. In this paper, we propose a novel adaptive aggregation scheme based on ordered weighted average (OWA) operators in contrast with the mainly used weighted mean scheme. Furthermore, besides presenting the theoretical aspects of our aggregation scheme, we develop two different concrete implementations to validate its suitability for video face recognition: Ordered weighted aggregation network (OWANet) and Weighted OWANet (WOWANet). Both algorithms are based on neural networks and are trainable through gradient descent in a classic supervised learning way. We conduct extensive experiments on YouTube Faces, COX Face and the IARPA Janus Benchmark A for evaluating recognition performance on verification and identification tasks. The experimentation process shows that both proposals achieve very competitive results in accuracy with respect to the existent state-of-the-art methods, while significantly reducing space and inference time.

The UU-Net: Reversible Face De-Identification for Visual Surveillance Video Footage

We propose a reversible face de-identification method for video surveillance data, where landmark-based techniques cannot be reliably used. Our solution generates a photorealistic de-identified stream that meets the data protection regulations and can be publicly released under minimal privacy concerns. Notably, such stream still encapsulates the information required to later reconstruct the original scene, which is useful for scenarios, such as crime investigation, where subjects identification is of most importance. Our learning process jointly optimizes two main components: 1) a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">public</i> module, that receives the raw data and generates the de-identified stream; and 2) a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">private</i> module, designed for security authorities, that receives the public stream and reconstructs the original data, disclosing the actual IDs of the subjects in a scene. The proposed solution is landmarks-free and uses a conditional generative adversarial network to obtain synthetic faces that preserve pose, lighting, background information and even facial expressions. Also, we keep full control over the set of soft facial attributes to be preserved/changed between the raw/de-identified data, which extends the range of applications for the proposed solution. Our experiments were conducted in three visual surveillance datasets (BIODI, MARS and P-DESTRE) plus one video face data set (YouTube Faces), showing highly encouraging results. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hugomcp/uu-net</uri> .

ST-VLAD: Video Face Recognition Based on Aggregated Local Spatial-Temporal Descriptors

How to integrate the temporal and spatial continuity information, when designing the video texture description operator, is crucial to realize video face recognition and facilitate video analysis and understanding, however, it has still yet to be properly addressed. In this paper, a novel video face recognition algorithm is proposed based on an aggregated local spatial-temporal descriptor (ST-VLAD), followed by a novel Fisher Criterion-based weight-learning method, which portrays the local information of the video more accurately, therefore largely improving the representation ability of description vectors. The proposed descriptor was tested on two representative databases, Honda/UCSD and YouTube Face database, achieving accuracies of 89.7% and 87.3%, respectively. The proposed method greatly outperformed the other existing state-of-art methods, suggesting a potential broad utility in the field of video face recognition.

Redundancy Removing Aggregation Network With Distance Calibration for Video Face Recognition

Attention-based techniques have been successfully used for rating image quality, and have been widely employed for set-based face recognition. Nevertheless, for video face recognition, where the base convolutional neural network (CNN) trained on large-scale data already provides discriminative features, fusing features with only predicted quality scores to generate representation are likely to cause duplicate sample dominant problem, and degrade performance correspondingly. To resolve the problem mentioned above, we propose a redundancy removing aggregation network (RRAN) for video face recognition. Compared with other quality-aware aggregation schemes, RRAN can take advantage of similarity information to tackle the noise introduced by redundant video frames. By leveraging metric learning, RRAN introduces a distance calibration scheme to align distance distributions of negative pairs of different video representations, which improves the accuracy under a uniform threshold. A series of experiments is conducted on multiple realistic data sets to evaluate the performance of RRAN, including YouTube Faces, IJB-A, and IJB-C. In comprehensive experiments, we demonstrate that our method can diminish the overall influence of poor quality components with large proportion in the video and further improve the overall recognition performance with individual difference. Specifically, RRAN achieves a 96.84% accuracy on YouTube Face, outperforming all existing aggregation schemes.

Discriminative metric learning for face verification using enhanced Siamese neural network

Although face verification algorithms have made great success under controlled conditions in recent years, there’s plenty of room at its performance under uncontrolled real-world due to lack of discriminative feature representation ability. From the perspective of metric learning, we proposed a context-aware based Siamese neural network (CASNN) to learn a simple yet powerful network for face verification task to enhance its discriminative feature representation ability. Firstly, a context-aware module is used to automatically focus on the key area of the input facial images without irrelevant background area. Then we design a Siamese network equipped with center-classification loss to compress intra-class features and enlarge between-class ones for discriminative metric learning. Finally, we propose a quantitative indicator named “D-score” to show the discriminative representation ability of the learnt features from different methods. The extensive experiments are conducted on LFW dataset, YouTube Face dataset (YTF) and real-world dataset. The results confirm that CASNN outperforms some state-of-the-art deep learning-based face verification methods.

FaceFilter: Face Identification with Deep Learning and Filter Algorithm

Although significant advances have been made recently in the field of face recognition, these have some limitations, especially when faces are in different poses or have different levels of illumination, or when the face is blurred. In this study, we present a system that can directly identify an individual under all conditions by extracting the most important features and using them to identify a person. Our method uses a deep convolutional network that is trained to extract the most important features. A filter is then used to select the most significant of these features by finding features greater than zero, storing their indices, and comparing the features of other identities with the same indices as the original image. Finally, the selected features of each identity in the dataset are subtracted from features of the original image to find the minimum number that refers to that identity. This method gives good results, as we only extract the most important features using the filter to recognize the face in different poses. We achieve state-of-the-art face recognition performance using only half of the 128 bytes per face. The system has an accuracy of 99.7% on the Labeled Faces in the Wild dataset and 94.02% on YouTube Faces DB.