Multi-view Face Research Articles

Open-Set Sheep Face Recognition in Multi-View Based on Li-SheepFaceNet

Deep learning-based sheep face recognition improves the efficiency and effectiveness of individual sheep recognition and provides technical support for the development of intelligent livestock farming. However, frequent changes within the flock and variations in facial features in different views significantly affect the practical application of sheep face recognition. In this study, we proposed the Li-SheepFaceNet, a method for open-set sheep face recognition in multi-view. Specifically, we employed the Seesaw block to construct a lightweight model called SheepFaceNet, which significantly improves both performance and efficiency. To enhance the convergence and performance of low-dimensional embedded feature learning, we used Li-ArcFace as the loss function. The Li-SheepFaceNet achieves an open-set recognition accuracy of 96.13% on a self-built dataset containing 3801 multi-view face images of 212 Ujumqin sheep, which surpasses other open-set sheep face recognition methods. To evaluate the robustness and generalization of our approach, we conducted performance testing on a publicly available dataset, achieving a recognition accuracy of 93.33%. Deploying Li-SheepFaceNet on an open-set sheep face recognition system enables the rapid and accurate identification of individual sheep, thereby accelerating the development of intelligent sheep farming.

Fully Deformable Network for Multiview Face Image Synthesis.

Photorealistic multiview face synthesis from a single image is a challenging problem. Existing works mainly learn a texture mapping model from the source to the target faces. However, they rarely consider the geometric constraints on the internal deformation arising from pose variations, which causes a high level of uncertainty in face pose modeling, and hence, produces inferior results for large pose variations. Moreover, current methods typically suffer from undesired facial details loss due to the adoption of the de-facto standard encoder-decoder architecture without any skip connections (SCs). In this article, we directly learn and exploit geometric constraints and propose a fully deformable network to simultaneously model the deformations of both landmarks and faces for face synthesis. Specifically, our model consists of two parts: a deformable landmark learning network (DLLN) and a gated deformable face synthesis network (GDFSN). The DLLN converts an initial reference landmark to an individual-specific target landmark as delicate pose guidance for face rotation. The GDFSN adopts a dual-stream structure, with one stream estimating the deformation of two views in the form of convolution offsets according to the source pose and the converted target pose, and the other leveraging the predicted deformation offsets to create the target face. In this way, individual-aware pose changes are explicitly modeled in the face generator to cope with geometric transformation, by adaptively focusing on pertinent regions of the source face. To compensate for offset estimation errors, we introduce a soft-gating mechanism for adaptive fusion between deformable features and primitive features. Additionally, a pose-aligned SC (PASC) is tailored to propagate low-level input features to the appropriate positions in the output features for further enhancing the facial details and identity preservation. Extensive experiments on six benchmarks show that our approach performs favorably against the state-of-the-arts, especially with large pose changes. Code is available at https://github.com/cschengxu/FDFace.

An approach-based machine learning and automated thermal images to predict the dark-cutting incidence in cattle management of healthcare supply chain

The healthcare supply chain is a network made up of various systems, processes, and elements that function and interact seamlessly to offer healthcare services and products. Food safety is an essential component of the healthcare supply chain. In the cattle industry, the healthcare supply chain contributes positively to the global economy through providing high-quality products such as milk and meat. Stress in cattle is one of main factor that cause low quality meat called “dark meat”. Numerous studies have been conducted on the development of different non-invasive methods based on Infrared Thermography Technology (IRT) to enhance the meat quality by detecting stress in cattle pre-slaughtering. These studies have the following issues: lack of automating in detecting body temperature of cattle and ignoring detecting stress with prediction dark meat incidence. The present study endeavors a new fully automated system for detecting stress, and dark meat incidence, incorporating the following new approaches: Multiview face detecting, Automatic eye localisation for detecting body temperature automatically employing computer vision and image processing, respectively. Furthermore, machine learning algorithms like Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB), and Decision Tree (DT) have been developed specifically for stress detection and the prediction of dark meat. To develop automated system, two forms of data were collected: statistical temperature and infrared thermal images. Infrared thermal images are used to develop Multiview face detection and Automatic eye localisation. Temperature data used to develop the machine learning model. Results reveal that Multiview face detection better than the current methods in term of Precision 0.99, Recall 0.91, F-score 0.95 with high True positive rate 0.90 and zero False-positive rate. Automatic eye localisation has high accuracy, with the following values: sensitivity 0.9780, precision 0.7212, F measure of 0.8024, and misclassification 0.0455. Lastly, results elaborate that the decision tree model can attain a notable level of accuracy in terms of specificity, recall, F-measure, and Area Under the Curve (AUC), all at an optimal rate of 98%.

Learning Dense Correspondence for NeRF-Based Face Reenactment

Face reenactment is challenging due to the need to establish dense correspondence between various face representations for motion transfer. Recent studies have utilized Neural Radiance Field (NeRF) as fundamental representation, which further enhanced the performance of multi-view face reenactment in photo-realism and 3D consistency. However, establishing dense correspondence between different face NeRFs is non-trivial, because implicit representations lack ground-truth correspondence annotations like mesh-based 3D parametric models (e.g., 3DMM) with index-aligned vertexes. Although aligning 3DMM space with NeRF-based face representations can realize motion control, it is sub-optimal for their limited face-only modeling and low identity fidelity. Therefore, we are inspired to ask: Can we learn the dense correspondence between different NeRF-based face representations without a 3D parametric model prior? To address this challenge, we propose a novel framework, which adopts tri-planes as fundamental NeRF representation and decomposes face tri-planes into three components: canonical tri-planes, identity deformations, and motion. In terms of motion control, our key contribution is proposing a Plane Dictionary (PlaneDict) module, which efficiently maps the motion conditions to a linear weighted addition of learnable orthogonal plane bases. To the best of our knowledge, our framework is the first method that achieves one-shot multi-view face reenactment without a 3D parametric model prior. Extensive experiments demonstrate that we produce better results in fine-grained motion control and identity preservation than previous methods.

Reconstructing facial motion across views using a multi-view face space.

Reconstructing facial motion across views using a multi-view face space.

Deep-Learning-Based Multiview RGBD Sensor System for 3-D Face Point Cloud Registration

Three-dimensional face analysis is now a hot research topic in computer vision. The acquisition of 3D faces requires acquiring multi-view face and obtaining them by point cloud registration. In this paper, a deep learning method is introduced in the multi-view RGBD depth sensor system to acquire matching point pairs of image data. Thus, 3D spatial point pairs are obtained. Then, the initial poses between the point clouds of multi-view faces are obtained by rigid body transformation estimation, and the coarse registration of the point clouds is completed. On this basis, for fine registration of multi-view 3D face point clouds using the Iterative Closest Point (ICP) algorithm, this paper proposes a method to convert partially overlapping point cloud registration into sub-region point cloud registration through facial region positioning. In this letter, the maximum value of RMSE for the experimental results of point cloud coarse registration is 2.26 mm with the mean and standard deviation of 1.18 ± 0.33 mm; the maximum value of RMSE for the experimental results of point cloud fine registration is 1.47 mm with the mean and standard deviation of 0.91 ± 0.14 mm. It is confirmed that the proposed method is very robust and effective for the registration of multi-view 3D face point clouds.

Convolutional Siamese neural network for few-shot multi-view face identification

Convolutional Siamese neural network for few-shot multi-view face identification

HeadPose-Softmax: Head pose adaptive curriculum learning loss for deep face recognition

Face recognition has been one of the most popular applications in the field of target detection. Currently, frontal faces can be easily detected, but multi-view face detection remains a difficult task because of various factors such as illumination, various poses, occlusions, and facial expressions. Margin-based loss functions are used to increase the feature margins between different classes, thus enhancing the discriminability of face recognition models, but the performance in face detection in complex scenes (e.g., high pitch angle face detection in surveillance environments) can be significantly degraded. Recently, the idea of a mining-based strategy to emphasize hard samples has been used to achieve good results in multi-view face detection. However, most of the existing methods do not explicitly emphasize samples based on their importance, resulting in the underutilization of hard samples. In this paper, we propose a curriculum learning loss function (HeadPose-Softmax) to classify the difficulty of a sample based on its facial pose, and embed the concept of curriculum learning into the loss function to implement a novel training strategy for deep face recognition. The loss function explicitly emphasizes the importance of the samples according to the different difficulty of each sample, which allows the model to make fuller use of hard samples, focus on learning pose invariant features, and improve the accuracy of the model in multi-view face detection tasks. Specifically, our HeadPose-Softmax dynamically adjusts the relative importance of the hard samples according to the pose angle of the face in the hard samples during the training phase. At each stage, different samples are assigned different importance according to their corresponding difficulty. Extensive experimental results under popular benchmarks show that our HeadPose-Softmax can enhance the accuracy of the model in multi-view face detection and outperform the state-of-the-art competitors.

WoT‐based Gaussian process multi‐view learning for real‐time monitoring and identification

With the rapid development of imaging and communication technology, monitoring equipment is becoming more and more popular, and how to realize the real‐time monitoring and identification of key figures has become an important research direction. However, the traditional monitoring and identification systems usually upload the collected data to the server directly, and then the monitoring staff manually identifies them, or uses the artificial intelligence algorithm to identify the data stored in the past, which may lead to missed detection or false detection, and the data uploading manner will take a lot of time, so that the final identification result does not have real‐time. To solve these issues, based on the Web of Things (WoT) technology, a Gaussian process multi‐view learning system is proposed in this paper. Specifically, the Gaussian process is first used to learn multi‐view fusion features, and the identification is carried out based on these fusion features. Secondly, according to the identification results, only the monitoring data of key figures or suspected key figures are uploaded to the server and displayed at the monitoring end, so as to effectively reduce the amount of data to be transmitted. Experimental results show that the proposed Gaussian process multi‐view learning algorithm can effectively identify multi view face images.

Face Verification Based on Deep Learning for Person Tracking in Hazardous Goods Factories

Person tracking in hazardous goods factories can provide a significant improvement in security and safety. This article proposes a face verification model which can be used to record travel paths for staff or related persons in the factory. As face images are captured from the dynamic crowd at entrance–exit gates of workshops, face verification is challenged by polymorphic faces, poor illumination and changing of a person’s pose. To adapt to this situation, a new face verification model is proposed, which is composed of two advanced deep learning neural network models. Firstly, MTCNN (Multi-Task Cascaded Convolutional Neural Network) is used to construct a face detector. Based on the SphereFace-20 network model, we have reconstructed a convolutional network architecture with the embedded Batch Normalization elements and the optimized network parameters. The new model, which is called the MDCNN, is used to extract efficient face features. A set of specific processing algorithms is used in the model to process polymorphic face images. The multi-view faces and various types of face images are used to train the models. The experimental results have demonstrated that the proposed model outperforms most existing methods on benchmark datasets such as the Labeled Faces in the Wild (LFW) and YouTube Face (YTF) datasets without multi-view (accuracy is 99.38% and 94.30%, respectively) and the CNBC/FERET datasets with multi-view (accuracy is 94.69%).

2D facial landmark localization method for multi-view face synthesis image using a two-pathway generative adversarial network approach.

One of the key challenges in facial recognition is multi-view face synthesis from a single face image. The existing generative adversarial network (GAN) deep learning methods have been proven to be effective in performing facial recognition with a set of pre-processing, post-processing and feature representation techniques to bring a frontal view into the same position in-order to achieve high accuracy face identification. However, these methods still perform relatively weak in generating high quality frontal-face image samples under extreme face pose scenarios. The novel framework architecture of the two-pathway generative adversarial network (TP-GAN), has made commendable progress in the face synthesis model, making it possible to perceive global structure and local details in an unsupervised manner. More importantly, the TP-GAN solves the problems of photorealistic frontal view synthesis by relying on texture details of the landmark detection and synthesis functions, which limits its ability to achieve the desired performance in generating high-quality frontal face image samples under extreme pose. We propose, in this paper, a landmark feature-based method (LFM) for robust pose-invariant facial recognition, which aims to improve image resolution quality of the generated frontal faces under a variety of facial poses. We therefore augment the existing TP-GAN generative global pathway with a well-constructed 2D face landmark localization to cooperate with the local pathway structure in a landmark sharing manner to incorporate empirical face pose into the learning process, and improve the encoder-decoder global pathway structure for better representation of facial image features by establishing robust feature extractors that select meaningful features that ease the operational workflow toward achieving a balanced learning strategy, thus significantly improving the photorealistic face image resolution. We verify the effectiveness of our proposed method on both Multi-PIE and FEI datasets. The quantitative and qualitative experimental results show that our proposed method not only generates high quality perceptual images under extreme poses but also significantly improves upon the TP-GAN results.

Evaluation of Deep Learning for Automatic Multi-View Face Detection in Cattle

Individual identification plays an important part in disease prevention and control, traceability of meat products, and improvement of agricultural false insurance claims. Automatic and accurate detection of cattle face is prior to individual identification and facial expression recognition based on image analysis technology. This paper evaluated the possibility of the cutting-edge object detection algorithm, RetinaNet, performing multi-view cattle face detection in housing farms with fluctuating illumination, overlapping, and occlusion. Seven different pretrained CNN models (ResNet 50, ResNet 101, ResNet 152, VGG 16, VGG 19, Densenet 121 and Densenet 169) were fine-tuned by transfer learning and re-trained on the dataset in the paper. Experimental results showed that RetinaNet incorporating the ResNet 50 was superior in accuracy and speed through performance evaluation, which yielded an average precision score of 99.8% and an average processing time of 0.0438 s per image. Compared with the typical competing algorithms, the proposed method was preferable for cattle face detection, especially in particularly challenging scenarios. This research work demonstrated the potential of artificial intelligence towards the incorporation of computer vision systems for individual identification and other animal welfare improvements.

Multiview Eye Localisation to Measure Cattle Body Temperature Based on Automated Thermal Image Processing and Computer Vision

Automatic and remote monitoring of body temperature through multi-view eye localisation could be very valuable to the cattle industry. Many studies have described that cattle's rising body temperature can be a sign of fever or local inflammation. Therefore, it would be beneficial for cattle producers to regularly measure cattle's body temperature to help identify at-risk cattle, make better informed management decisions, and maintain their mob's commercial value. However, under commercial conditions, it can be difficult to accurately measure the temperature of large mobs of cattle in these production environments. Traditionally, methods used to measure body temperature were invasive and time-consuming, and there has been a demand for an alternative method for body temperature measurement. Hence, automated processes can be used to measure cattle's body temperature efficiently and accurately, and it would be of great benefit to the cattle industry. The infrared Thermography Technique (IRT) has the capability to measure the body temperature of cattle by analysing an infrared thermal image of the eye region in real-time. IRT has played a vital role in the medical sector by developing automated systems for fever, infection detection, and musculoskeletal disorders. Most of the previously proposed eye localization methods used frontal position only in humans, and manual localisation in cattle. Our contributions include proposing thermal image processing for eye segmentation and a computer vision method for multi-view face detection in cattle. In the computer vision method, three classifiers were created for multi-view face detection, which uses Histogram Oriented Gradient (HOG) as features and Support vector machine (SVM) as the classifier. While in image processing, a new automated eye segmentation method based on thresholding techniques is proposed. Proposed methods work together in the indoor environment to localize the eye region of cattle by measuring temperature from different orientations, angles, and positions. The paper results showed that the proposed method has high accuracy, with the following average sensitivity of sensitivity 0.9780, precision 0.7212, F measure of 0.8024, and misclassification 0.0455.

A novel eye center localization method for multiview faces

Existing studies on eye center localization have mostly applied localization methods on faces viewed from frontal angles or faces with small yaw-rotation angles. This study proposed a novel eye center localization method that can effectively localize the eye centers under various situations on multiview faces with large yaw-rotation angles (from +67.5° to −67.5°). First, this study developed a multiview face detector that can be applied to large yaw angles and can flexibly detect and precisely capture the face region. The face detector can facilitate the generation of satisfactory results by a complete representation generative adversarial network (CR-GAN). Because a large yaw-rotation angle can substantially reduce the completeness of eye representation in an image or even cause the eyes to disappear within the face image, this study used a CR-GAN to produce frontal face images to solve the problem of incomplete representation in multiview face images. Furthermore, this study proposed a new iris-ripple filter to increase the accuracy and robustness of gradient localization. Finally, a new depth corresponding-points conversion method was proposed to automatically estimate the rotation variable between two faces and the conversion relationship between the eye centers. This method can effectively solve the problem of instability resulting from a CR-GAN during eye generation and can ensure localization accuracy for eyeballs with subtle changes. According to the experimental results, compared with other advanced methods, the proposed method exhibited higher accuracy and robustness in eye center localization when applied to images obtained from four databases that involved various challenging situations, including large yaw-rotation angle, illumination change, head pose change, gaze interaction, and complete occlusion of eyes.

Multi-view face generation via unpaired images

Multi-view face generation from a single image is an essential and challenging problem. Most of the existing methods need to use paired images when training models. However, collecting and labeling large-scale paired face images could lead to high labor and time cost. In order to address this problem, multi-view face generation via unpaired images is proposed in this paper. To avoid using paired data, the encoder and discriminator are trained, so that the high-level abstract features of the identity and view of the input image are learned by the encoder, and then, these low-dimensional data are input into the generator, so that the realistic face image can be reconstructed by the training generator and discriminator. During testing, multiple one-hot vectors representing the view are imposed to the identity representation and the generator is employed to map them to high-dimensional data, respectively, which can generate multi-view images while preserving the identity features. Furthermore, to reduce the number of used labels, semi-supervised learning is used in the model. The experimental results show that our method can produce photo-realistic multi-view face images with a small number of view labels, and makes a useful exploration for the synthesis of face images via unpaired data and very few labels.

Facial memory ability and self-awareness in patients with temporal lobe epilepsy after anterior temporal lobectomy.

Anterior temporal lobectomy (ATL) is the most common surgical treatment for drug-resistant temporal lobe epilepsy (TLE). Right ATL has been reported to reduce facial memory ability in patients with TLE, as indicated by poor performance on the Warrington Recognition Memory Test for Faces (RMF), which is commonly used to evaluate visual memory in these patients. However, little is known about whether patients with TLE exhibit difficulties in identifying faces in daily life after ATL. The aim of this study was to investigate facial memory ability and self-awareness of face identification difficulties in patients with TLE after ATL. Sixteen patients with TLE after right ATL, 14 patients with TLE after left ATL, and 29 healthy controls were enrolled in this study. We developed the multiview face recognition test (MFRT), which comprises a learning phase (one or three frontal face images without external facial feature information) and a recognition phase (frontal, oblique, or noise-masked face images). Facial memory abilities were examined in all participants using the MFRT and RMF, and self-awareness of difficulties in face identification was evaluated using the 20-item prosopagnosia index (PI20), which has been widely used to assess developmental prosopagnosia. The MFRT performance in patients with TLE after ATL was significantly worse than that in healthy controls regardless of the resected side, whereas the RMF scores in patients with TLE were significantly worse than those in healthy controls only after right ATL. The MFRT performance in patients with TLE after both left and right ATL was more influenced by working memory load than that in healthy controls. The PI20 scores revealed that patients with TLE after left ATL were aware of their difficulties in identifying faces. These findings suggest that patients with TLE not only after right ATL but also after left ATL might have difficulties in face identification.

Frontal and non-frontal face detection using deep neural networks (DNN)

Face recognition has always been one of the most searched and popular applications of object detection, starting from the early seventies. Facial recognition is used for access control, authentication, fraud detection, surveillance, and by individuals to unlock their devices. The less intrusive and robustness of the face detection systems, make it better than the fingerprint scanner and iris scanner. The frontal face can be easily detected, but multi-view face detection remains a difficult task, due to various factors like illumination, various poses, occlusions, and facial expressions. In this paper, we propose a Deep Neural Network (DNN) based approach to improve the accuracy of detection of the face. We show that Deep Neural Networks algorithms have better accuracy than traditional face detection algorithms for multi-view face detection. The Deep Neural Network (DNN) gives more precise and accurate results, as the DNN model is trained with large datasets and, the model learns the best features from the dataset.

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

In order to overcome the problems of large recognition error and low recognition accuracy in the existing face pose recognition models, the paper proposes and constructs a new multi-view face pose recognition model based on typical correlation analysis algorithm. First, the AdaBoost algorithm is used to realise multi-pose face detection and positioning. Secondly, the face image is pre-processed, including image greying, image denoising, and face image geometric normalisation, and then the typical correlation analysis algorithm is used to extract face features. Finally, multi-view facial gesture recognition is realised through convolutional neural network. Experimental results show that, compared with the traditional recognition model, the recognition accuracy of the constructed model is greatly improved, and the average accuracy (mAP) is 96.334%, which proves that the recognition performance of the constructed model is better.

Erratum to "Multi-View Face Synthesis via Progressive Face Flow".

In the above article [1], unfortunately, Fig. 5 was not displayed correctly with many empty images. The correct version is supplemented here.