Fusion Of Facial Features Research Articles

Multi‐parameter fusion driver fatigue detection method based on facial fatigue features

AbstractFatigued driving is one of the main causes of traffic accidents. In order to improve the detection speed of fatigue driving recognition, this paper proposes a driver fatigue detection method based on multi‐parameter fusion of facial features. It uses a cascaded Adaboost object classifier to detect faces in video streams. The DliB library is employed for facial key point detection, which locates the driver's eyes and mouth to determine their states. The eye aspect ratio (EAR) is calculated to detect eye closure, and the mouth aspect ratio (MAR) is calculated to detect yawning frequency and count. The detected percentage of eye closure (PERCLOS) value is combined with yawning frequency and count, and a multi‐feature fusion approach is used for fatigue detection. Experimental results show that the accuracy of blink detection is 91% and the accuracy of yawn detection is 96.43%. Furthermore, compared to the models mentioned in the comparative experiments, this model achieves two to four times faster detection times while maintaining accuracy.

Device-edge collaborative occluded face recognition method based on cross-domain feature fusion

The lack of facial features caused by wearing masks degrades the performance of facial recognition systems. Traditional occluded face recognition methods cannot integrate the computational resources of the edge layer and the device layer. Besides, previous research fails to consider the facial characteristics including occluded and unoccluded parts. To solve the above problems, we put forward a device-edge collaborative occluded face recognition method based on cross-domain feature fusion. Specifically, the device-edge collaborative face recognition architecture gets the utmost out of maximizes device and edge resources for real-time occluded face recognition. Then, a cross-domain facial feature fusion method is presented which combines both the explicit domain and the implicit domain facial. Furthermore, a delay-optimized edge recognition task scheduling method is developed that comprehensively considers the task load, computational power, bandwidth, and delay tolerance constraints of the edge. This method can dynamically schedule face recognition tasks and minimize recognition delay while ensuring recognition accuracy. The experimental results show that the proposed method achieves an average gain of about 21% in recognition latency, while the accuracy of the face recognition task is basically the same compared to the baseline method.

Detecting depression based on facial cues elicited by emotional stimuli in video

Recently, depression research has received considerable attention and there is an urgent need for objective and validated methods to detect depression. Depression detection based on facial expressions may be a promising adjunct to depression detection due to its non-contact nature. Stimulated facial expressions may contain more information that is useful in detecting depression than natural facial expressions. To explore facial cues in healthy controls and depressed patients in response to different emotional stimuli, facial expressions of 62 subjects were collected while watching video stimuli, and a local face reorganization method for depression detection is proposed. The method extracts the local phase pattern features, facial action unit (AU) features and head motion features of a local face reconstructed according to facial proportions, and then fed into the classifier for classification. The classification accuracy was 76.25%, with a recall of 80.44% and a specificity of 83.21%. The results demonstrated that the negative video stimuli in the single-attribute stimulus analysis were more effective in eliciting changes in facial expressions in both healthy controls and depressed patients. Fusion of facial features under both neutral and negative stimuli was found to be useful in discriminating between healthy controls and depressed individuals. The Pearson correlation coefficient (PCC) showed that changes in the emotional stimulus paradigm were more strongly correlated with changes in subjects’ facial AU when exposed to negative stimuli compared to stimuli of other attributes. These results demonstrate the feasibility of our proposed method and provide a framework for future work in assisting diagnosis.

Facial feature fusion convolutional neural network for driver fatigue detection

Driver fatigue detection is critical for preventing traffic accidents. However, in real driving scenarios, low-quality input situations often occur due to occlusion, large-turn head deformation, and inconsistent eye states, which introduce noise and degrade the performance of single-stream networks. To address this, we propose a multi-stream facial feature fusion convolution neural network (FFF–CNN) that enhances performance on low-quality inputs. The FFF-CNN includes a global face stream and two local eye streams, with feature fusion used for eye state classification. To facilitate feature fusion, we implemented two preprocessing modules, the feature attention module (FAM) and the stream interaction module (SIM). FAM emphasizes important features in the global face stream, while SIM corrects information from the two local eye streams. The proposed FFF-CNN achieved good performance on both our dataset and the public Closed Eyes in the Wild (CEW) dataset. Compared to single-stream networks, the FFF-CNN improved performance by 4.95% and 4.27% on our dataset and the CEW dataset, respectively. Furthermore, it achieved state-of-the-art performance on the CEW dataset with an accuracy of 98.35%. The effectiveness of the proposed FAM and SIM has been verified by experiments. Notably, our experimental results show that the proposed FFF-CNN effectively improved fatigue detection performance on low-quality input, approaching the performance on normal situations.

Joint Facial Action Unit Detection and Feature Fusion: A Multi-conditional Learning Approach.

Automated analysis of facial expressions can benefit many domains, from marketing to clinical diagnosis of neurodevelopmental disorders. Facial expressions are typically encoded as a combination of facial muscle activations, i.e., action units. Depending on context, these action units co-occur in specific patterns, and rarely in isolation. Yet, most existing methods for automatic action unit detection fail to exploit dependencies among them, and the corresponding facial features. To address this, we propose a novel multi-conditional latent variable model for simultaneous fusion of facial features and joint action unit detection. Specifically, the proposed model performs feature fusion in a generative fashion via a low-dimensional shared subspace, while simultaneously performing action unit detection using a discriminative classification approach. We show that by combining the merits of both approaches, the proposed methodology outperforms existing purely discriminative/generative methods for the target task. To reduce the number of parameters, and avoid overfitting, a novel Bayesian learning approach based on Monte Carlo sampling is proposed, to integrate out the shared subspace. We validate the proposed method on posed and spontaneous data from three publicly available datasets (CK+, DISFA and Shoulder-pain), and show that both feature fusion and joint learning of action units leads to improved performance compared to the state-of-the-art methods for the task.

Driver Alertness Monitoring Using Fusion of Facial Features and Bio-Signals

Driver drowsiness is among the leading causal factors in traffic accidents occurring worldwide. This paper describes a method to monitor driver safety by analyzing information related to fatigue using two distinct methods: eye movement monitoring and bio-signal processing. A monitoring system is designed in Android-based smartphone where it receives sensory data via wireless sensor network and further processes the data to indicate the current driving aptitude of the driver. It is critical that several sensors are integrated and synchronized for a more realistic evaluation of the driver behavior. The sensors applied include a video sensor to capture the driver image and a bio-signal sensor to gather the driver photoplethysmograph signal. A dynamic Bayesian network framework is used for the driver fatigue evaluation. A warning alarm is sounded if driver fatigue is believed to reach a defined threshold. The manifold testing of the system demonstrates the practical use of multiple features, particularly with discrete methods, and their fusion enables a more authentic and ample fatigue detection.

Selection and fusion of facial features for face recognition

This paper proposes and investigates a facial feature selection and fusion technique for improving the classification accuracy of face recognition systems. The proposed technique is novel in terms of feature selection and fusion processes. It incorporates neural networks and genetic algorithms for the selection and classification of facial features. The proposed technique is evaluated by using the separate facial region features and the combined features. The combined features outperform the separate facial region features in the experimental investigation. A comprehensive comparison with other existing face recognition techniques on FERET benchmark database is included in this paper. The proposed technique has produced 94% classification accuracy, which is a significant improvement and best classification accuracy among the published results in the literature.