YouTube Faces Dataset Research Articles

Neural dynamics for improving optimiser in deep learning with noise considered

AbstractAs deep learning evolves, neural network structures become increasingly sophisticated, bringing a series of new optimisation challenges. For example, deep neural networks (DNNs) are vulnerable to a variety of attacks. Training neural networks under privacy constraints is a method to alleviate privacy leakage, and one way to do this is to add noise to the gradient. However, the existing optimisers suffer from weak convergence in the presence of increased noise during training, which leads to a low robustness of the optimiser. To stabilise and improve the convergence of DNNs, the authors propose a neural dynamics (ND) optimiser, which is inspired by the zeroing neural dynamics originated from zeroing neural networks. The authors first analyse the relationship between DNNs and control systems. Then, the authors construct the ND optimiser to update network parameters. Moreover, the proposed ND optimiser alleviates the non‐convergence problem that may be suffered by adding noise to the gradient from different scenarios. Furthermore, experiments are conducted on different neural network structures, including ResNet18, ResNet34, Inception‐v3, MobileNet, and long and short‐term memory network. Comparative results using CIFAR, YouTube Faces, and R8 datasets demonstrate that the ND optimiser improves the accuracy and stability of DNNs under noise‐free and noise‐polluted conditions. The source code is publicly available at https://github.com/LongJin‐lab/ND.

ALMEGA-VIR: face video retrieval system

ABSTRACT The limitations of Content Based Video Retrieval (CBVR), such as large pools of video data, selection of features, limited processing capacity, and content-related issues, can be overcome by Deep Belief Neural Networks (DBNs). The search engine does the processing, and the results are effectively returned to the users. The deep learning model also has some comparable challenges to be solved at various levels. Typically, the selection of frames is one of the most essential tasks for a face retrieval system which is efficiently accomplished by using deep learning models. Also, the performance of recognition models highly depends on the selection of frames that include both low-level and high-level features from the given video sequence. This paper proposes ALPHA-TO-OMEGA (ALMEGA-VIR) Visual Information Retrieval scheme that analyzes the contents of the video file for a particular face and retrieves the user's relevant videos. VIR aims to solve the challenge of locating relevant pictures and videos based on a query. VIR can be based on text-based search, content-based search, and Mixed based search. VIR here focuses on content-based search approach. The YouTube Celebrities (YTC) dataset, the Youtube Face (YTF) dataset, and Honda / UCSD dataset are used in this work. The used dataset addresses issues such as illumination changes, occlusion, rotation, and scale. We propose an ALMEGA-VIR framework for visual novelty face identification for specific investigation and evaluation. The classification accuracy of the proposed ALMEGA-VIR model shows that 4% is more effective than traditional classifier algorithms such as SVM, Naïve Bayesian classifier, MLP and Random Forest classifier. A comparative study is also implemented with other existing works to check the superiority of the proposed method.

Attendance Compilation by Facial Recognition Methods of Image Processing: A Review

Abstract: In recent years, researchers have continued to refine and improve deep learning-based approaches to image processing, as well as exploring new areas such as generative adversarial networks (GANs) and reinforcement learning. This paper provides a comprehensive survey of deep learning-based methods for face recognition, including CNN-based models, auto encoder models, and hybrid models. These papers demonstrate the effectiveness of deep convolutional neural networks (CNNs) in face detection, recognition, and attendance compilation, achieving state-of-the-art accuracy on several benchmark datasets, including LFW, YouTube Faces, and YTF datasets. The Efficient Net model is a family of CNNs that achieves state of the art accuracy on multiple image recognition benchmarks while being significantly smaller and faster than previous models. The Arc Face loss function is used for facial landmark detection and gender classification in facial images.

Face Verification Based on Deep Learning for Person Tracking in Hazardous Goods Factories

Person tracking in hazardous goods factories can provide a significant improvement in security and safety. This article proposes a face verification model which can be used to record travel paths for staff or related persons in the factory. As face images are captured from the dynamic crowd at entrance–exit gates of workshops, face verification is challenged by polymorphic faces, poor illumination and changing of a person’s pose. To adapt to this situation, a new face verification model is proposed, which is composed of two advanced deep learning neural network models. Firstly, MTCNN (Multi-Task Cascaded Convolutional Neural Network) is used to construct a face detector. Based on the SphereFace-20 network model, we have reconstructed a convolutional network architecture with the embedded Batch Normalization elements and the optimized network parameters. The new model, which is called the MDCNN, is used to extract efficient face features. A set of specific processing algorithms is used in the model to process polymorphic face images. The multi-view faces and various types of face images are used to train the models. The experimental results have demonstrated that the proposed model outperforms most existing methods on benchmark datasets such as the Labeled Faces in the Wild (LFW) and YouTube Face (YTF) datasets without multi-view (accuracy is 99.38% and 94.30%, respectively) and the CNBC/FERET datasets with multi-view (accuracy is 94.69%).

Discriminative metric learning for face verification using enhanced Siamese neural network

Although face verification algorithms have made great success under controlled conditions in recent years, there’s plenty of room at its performance under uncontrolled real-world due to lack of discriminative feature representation ability. From the perspective of metric learning, we proposed a context-aware based Siamese neural network (CASNN) to learn a simple yet powerful network for face verification task to enhance its discriminative feature representation ability. Firstly, a context-aware module is used to automatically focus on the key area of the input facial images without irrelevant background area. Then we design a Siamese network equipped with center-classification loss to compress intra-class features and enlarge between-class ones for discriminative metric learning. Finally, we propose a quantitative indicator named “D-score” to show the discriminative representation ability of the learnt features from different methods. The extensive experiments are conducted on LFW dataset, YouTube Face dataset (YTF) and real-world dataset. The results confirm that CASNN outperforms some state-of-the-art deep learning-based face verification methods.

Multimodal facial biometrics recognition: Dual-stream convolutional neural networks with multi-feature fusion layers

Facial recognition for surveillance applications still remains challenging in uncontrolled environments, especially with the appearances of masks/veils and different ethnicities effects. Multimodal facial biometrics recognition becomes one of the major studies to overcome such scenarios. However, to cooperate with multimodal facial biometrics, many existing deep learning networks rely on feature concatenation or weight combination to construct a representation layer to perform its desired recognition task. This concatenation is often inefficient, as it does not effectively cooperate with the multimodal data to improve on recognition performance. Therefore, this paper proposes using multi-feature fusion layers for multimodal facial biometrics, thereby leading to significant and informative data learning in dual-stream convolutional neural networks. Specifically, this network consists of two progressive parts with distinct fusion strategies to aggregate RGB data and texture descriptors for multimodal facial biometrics. We demonstrate that the proposed network offers a discriminative feature representation and benefits from the multi-feature fusion layers for an accuracy-performance gain. We also introduce and share a new dataset for multimodal facial biometric data, namely the Ethnic-facial dataset for benchmarking. In addition, four publicly accessible datasets, namely AR, FaceScrub, IMDB_WIKI, and YouTube Face datasets are used to evaluate the proposed network. Through our experimental analysis, the proposed network outperformed several competing networks on these datasets for both recognition and verification tasks.

A deep learning approach for face recognition based on angularly discriminative features

Face recognition in digital images or video frames has several real-world applications in the modern zone of computer vision. Loss function plays a vital role in deep face recognition. Recently, several loss functions have been proposed in classification techniques to reduce the number of model errors. Among several loss functions, softmax loss implemented either multiplicative angular or additive cosine margin. These individual margins have less capacity to reduce the model`s errors. To fill this gap; we proposed a hybrid angularly discriminative features by combining multiplicative angular and additive cosine margin to improve the efficiency of angular softmax loss and large margin cosine. We trained proposed model using CASIA-WebFace dataset and testing has been performed on Labeled Face in the Wild (LFW), YouTube Faces (YTF), VGGFace1 and VGGFace2. The experimental result shows 99.77% accuracy on LFW dataset whereas 96.40% accuracy achieved on YTF dataset which is higher than the existing similar state-of-the-art techniques.

Face Set Recognition

In this paper we present a Cluster Aggregation Network (CAN) for face set recognition. This network takes a set of face images, which could be either face videos or clusters with a different number of face images as its input, and then it is able to produce a compact and fixed-dimensional feature representation for the face set for the purpose of recognition. The whole network is made up of two modules, among which the first one is a face feature embedding module and the second one is the face feature aggregation module. The first module is a deep Convolutional Neural Network (CNN) which maps each of the face images to a fixed-dimensional vector. The second module is also a CNN which is trained to be able to automatically assess the quality of input face images and thus assign various weights to the images’ corresponding feature vectors. Then the one aggregated feature vector representing the input set is formed inside the convex hull formed by the input single face image features. Due to the mechanism that quality assessment is invariant to the order of one image in a set and the number of images in the set, the aggregation is invariant to these factors. Our CAN is trained with standard classification loss without any other supervision information and we found that our network is automatically attracted to high quality face images, while repelling low quality images, such as blurred, blocked, and non-frontal face images. We trained our networks with CASIA and YouTube Face datasets and the experiments on IJB-C video face recognition benchmark show that our method outperforms the current state-of-the-art feature aggregation methods and our challenging baseline aggregation method.

Template adaptation for face verification and identification

Face recognition performance evaluation has traditionally focused on one-to-one verification, popularized by the Labeled Faces in the Wild data set [1] for imagery and the YouTubeFaces data set [2] for videos. In contrast, the newly released IJB-A face recognition data set [3] unifies evaluation of one-to-many face identification with one-to-one face verification over templates, or sets of imagery and videos for a subject. In this paper, we study the problem of template adaptation, a form of transfer learning to the set of media in a template. Extensive performance evaluations on IJB-A show a surprising result, that perhaps the simplest method of template adaptation, combining deep convolutional network features with template specific linear SVMs, outperforms the state-of-the-art by a wide margin. We study the effects of template size, negative set construction and classifier fusion on performance, then compare template adaptation to convolutional networks with metric learning, 2D and 3D alignment. Our unexpected conclusion is that these other methods, when combined with template adaptation, all achieve nearly the same top performance on IJB-A for template-based face verification and identification.

A joint loss function for deep face recognition

Convolutional neural networks (CNNs) have been widely used in computer vision community, and significantly improving the state-of-the-art. How to train an intra-class variant and inter-class discriminative feature is a central topic in face recognition. This paper proposes to learn an effective feature from face images by a joint loss function which combines the hard sample triplet (HST) and the absolute constraint triplet (ACT) loss, under the criteria that a maximum intra-class distance should be smaller than any inter-class distance. With the joint supervision of HST and ACT loss, CNNs is enable to learn discriminative features to improve face recognition performance. Experiments on labeled faces in the wild, IARPA Janus Benchmark (IJB-A) and YouTube Faces datasets achieve a comparable or superior performance to the state-of-the-arts.

Investigating Nuisances in DCNN-based Face Recognition.

Face recognition "in the wild" has been revolutionized by the deployment of deep learning based approaches. In fact, it has been extensively demonstrated that Deep Convolutional Neural Networks (DCNNs) are powerful enough to overcome most of the limits that affected face recognition algorithms based on hand-crafted features. These include variations in illumination, pose, expression and occlusion, to mention some. The DCNNs discriminative power comes from the fact that low- and high-level representations are learned directly from the raw image data. As a consequence, we expect the performance of a DCNN to be influenced by the characteristics of the image/video data that are fed to the network, and their preprocessing. In this work, we present a thorough analysis of several aspects that impact on the use of DCNN for face recognition. The evaluation has been carried out from two main perspectives: the network architecture and the similarity measures used to compare deeply learned features; the data (source and quality) and their preprocessing (bounding box and alignment). Results obtained on the IJB-A, MegaFace, UMDFaces and YouTube Faces datasets indicate viable hints for designing, training and testing DCNNs. Taking into account the outcomes of the experimental evaluation, we show how competitive performance with respect to the state-of-the-art can be reached even with standard DCNN architectures and pipeline.

A real-time and unsupervised face re-identification system for human-robot interaction

In the context of Human-Robot Interaction (HRI), face Re-Identification (face Re-ID) aims to verify if certain detected faces have already been observed by robots. The ability of distinguishing between different users is crucial in social robots as it will enable the robot to tailor the interaction strategy toward the users’ individual preferences. So far face recognition research has achieved great success, however little attention has been paid to the realistic applications of Face Re-ID in social robots. In this paper, we present an effective and unsupervised face Re-ID system which simultaneously re-identifies multiple faces for HRI. This Re-ID system employs Deep Convolutional Neural Networks to extract features, and an online clustering algorithm to determine the face's ID. Its performance is evaluated on two datasets: the TERESA video dataset collected by the TERESA robot, and the YouTube Face Dataset (YTF Dataset). We demonstrate that the optimised combination of techniques achieves an overall 93.55% accuracy on TERESA dataset and an overall 90.41% accuracy on YTF dataset. We have implemented the proposed method into a software module in the HCI^2 Framework [1] for it to be further integrated into the TERESA robot [2], and has achieved real-time performance at 10–26 Frames per second.

Dynamic Texture Comparison Using Derivative Sparse Representation: Application to Video-Based Face Recognition

Video-based face, expression, and scene recognition are fundamental problems in human–machine interaction, especially when there is a short-length video. In this paper, we present a new derivative sparse representation approach for face and texture recognition using short-length videos. First, it builds local linear subspaces of dynamic texture segments by computing spatiotemporal directional derivatives in a cylinder neighborhood within dynamic textures. Unlike traditional methods, a nonbinary texture coding technique is proposed to extract high-order derivatives using continuous circular and cylinder regions to avoid aliasing effects. Then, these local linear subspaces of texture segments are mapped onto a Grassmann manifold via sparse representation. A new joint sparse representation algorithm is developed to establish the correspondences of subspace points on the manifold for measuring the similarity between two dynamic textures. Extensive experiments on the Honda/UCSD, the CMU motion of body, the YouTube, and the DynTex datasets show that the proposed method consistently outperforms the state-of-the-art methods in dynamic texture recognition, and achieved the encouraging highest accuracy reported to date on the challenging YouTube face dataset. The encouraging experimental results show the effectiveness of the proposed method in video-based face recognition in human–machine system applications.

Unconstrained Still/Video-Based Face Verification with Deep Convolutional Neural Networks

Over the last 5 years, methods based on Deep Convolutional Neural Networks (DCNNs) have shown impressive performance improvements for object detection and recognition problems. This has been made possible due to the availability of large annotated datasets, a better understanding of the non-linear mapping between input images and class labels as well as the affordability of GPUs. In this paper, we present the design details of a deep learning system for unconstrained face recognition, including modules for face detection, association, alignment and face verification. The quantitative performance evaluation is conducted using the IARPA Janus Benchmark A (IJB-A), the JANUS Challenge Set 2 (JANUS CS2), and the Labeled Faces in the Wild (LFW) dataset. The IJB-A dataset includes real-world unconstrained faces of 500 subjects with significant pose and illumination variations which are much harder than the LFW and Youtube Face datasets. JANUS CS2 is the extended version of IJB-A which contains not only all the images/frames of IJB-A but also includes the original videos. Some open issues regarding DCNNs for face verification problems are then discussed.

Joint prototype and metric learning for image set classification: Application to video face identification

In this paper, we address the problem of image set classification, where each set contains a different number of images acquired from the same subject. In most of the existing literature, each image set is modeled using all its available samples. As a result, the corresponding time and storage costs are high. To address this problem, we propose a joint prototype and metric learning approach. The prototypes are learned to represent each gallery image set using fewer samples without affecting the recognition performance. A Mahalanobis metric is learned simultaneously to measure the similarity between sets more accurately. In particular, each gallery set is represented as a regularized affine hull spanned by the learned prototypes. The set-to-set distance is optimized via updating the prototypes and the Mahalanobis metric in an alternating manner. To highlight the importance of representing image sets using fewer samples, we analyzed the corresponding test time complexity with respect to the number of images used per set. Experimental results using YouTube Celebrity, YouTube Faces, and ETH-80 datasets illustrate the efficiency on the task of video face recognition, and object categorization.

Localized Multifeature Metric Learning for Image-Set-Based Face Recognition

This paper presents a new approach to image-set-based face recognition, where each training and testing example is a set of face images captured from varying poses, illuminations, expressions, and resolutions. While a number of image set based face recognition methods have been proposed in recent years, most of them model each face image set as a single linear subspace or as the union of linear subspaces, which may lose some discriminative information for face image set representation. To address this shortcoming, we propose exploiting statistics information as feature representations for face image sets and develop a localized multikernel metric learning algorithm to effectively combine different statistics for recognition. Moreover, we propose a localized multikernel multimetric learning method to jointly learn multiple feature-specific distance metrics in the kernel spaces, one for each statistic feature, to better exploit complementary information for recognition. Our methods achieve state-of-the-art performance on four widely used video face datasets including the Honda, MoBo, YouTube Celebrities, and YouTube Face datasets.

Reconstruction-Based Metric Learning for Unconstrained Face Verification

In this paper, we propose a reconstruction-based metric learning method to learn a discriminative distance metric for unconstrained face verification. Unlike conventional metric learning methods, which only consider the label information of training samples and ignore the reconstruction residual information in the learning procedure, we apply a reconstruction criterion to learn a discriminative distance metric. For each training example, the distance metric is learned by enforcing a margin between the interclass sparse reconstruction residual and interclass sparse reconstruction residual, so that the reconstruction residual of training samples can be effectively exploited to compute the between-class and within-class variations. To better use multiple features for distance metric learning, we propose a reconstruction-based multimetric learning method to collaboratively learn multiple distance metrics, one for each feature descriptor, to remove uncorrelated information for recognition. We evaluate our proposed methods on the Labelled Faces in the Wild (LFW) and YouTube face data sets and our experimental results clearly show the superiority of our methods over both previous metric learning methods and several state-of-the-art unconstrained face verification methods.