Percentage Of Eyelid Closure Research Articles

Detection of fatigue driving behavior based on facial expression

The scene environment in the cab is complex and changeable, which is easy to be disturbed by the changes of weather, light and other conditions, which brings many difficulties to the driver’s fatigue driving detection. This paper mainly analyzes the specific characteristics of the driving indoor environment, uses some technologies in the field of computer vision and image processing to improve the image quality, uses the Adaboost face detection algorithm based on Haar-like features, to detect and locate the driver’s face in the cockpit, and reduces the face detection area to determine whether the driver looks left and right or bowed. Then, on the basis of detecting the face, the method of gray scale integral projection is adopted to locate the eyes and the mouth, and the Bezier curve is used to fit the contour of the eyes and the mouth, and to judge the driver’s eye opening and the mouth opening by comprehensively analyzing the vertical and horizontal ratio. Later, ocular fatigue analysis was performed by using the PERCLOS(Percentage of Eyelid Closure over the Pupil over Time) criteria and by eyeblink frequency, and mouth fatigue analysis was also performed by the frequency of yawning.

Research on Fatigued-Driving Detection Method by Integrating Lightweight YOLOv5s and Facial 3D Keypoints.

In response to the problem of high computational and parameter requirements of fatigued-driving detection models, as well as weak facial-feature keypoint extraction capability, this paper proposes a lightweight and real-time fatigued-driving detection model based on an improved YOLOv5s and Attention Mesh 3D keypoint extraction method. The main strategies are as follows: (1) Using Shufflenetv2_BD to reconstruct the Backbone network to reduce parameter complexity and computational load. (2) Introducing and improving the fusion method of the Cross-scale Aggregation Module (CAM) between the Backbone and Neck networks to reduce information loss in shallow features of closed-eyes and closed-mouth categories. (3) Building a lightweight Context Information Fusion Module by combining the Efficient Multi-Scale Module (EAM) and Depthwise Over-Parameterized Convolution (DoConv) to enhance the Neck network's ability to extract facial features. (4) Redefining the loss function using Wise-IoU (WIoU) to accelerate model convergence. Finally, the fatigued-driving detection model is constructed by combining the classification detection results with the thresholds of continuous closed-eye frames, continuous yawning frames, and PERCLOS (Percentage of Eyelid Closure over the Pupil over Time) of eyes and mouth. Under the premise that the number of parameters and the size of the baseline model are reduced by 58% and 56.3%, respectively, and the floating point computation is only 5.9 GFLOPs, the average accuracy of the baseline model is increased by 1%, and the Fatigued-recognition rate is 96.3%, which proves that the proposed algorithm can achieve accurate and stable real-time detection while lightweight. It provides strong support for the lightweight deployment of vehicle terminals.

Efficient and Robust Driver Fatigue Detection Framework Based on the Visual Analysis of Eye States

Fatigue detection based on vision is widely employed in vehicles due to its real-time and reliable detection results. With the coronavirus disease (COVID-19) outbreak, many proposed detection systems based on facial characteristics would be unreliable due to the face covering with the mask. In this paper, we propose a robust visual-based fatigue detection system for monitoring drivers, which is robust regarding the coverings of masks, changing illumination and head movement of drivers. Our system has three main modules: face key point alignment, fatigue feature extraction and fatigue measurement based on fused features. The innovative core techniques are described as follows: (1) a robust key point alignment algorithm by fusing global face information and regional eye information, (2) dynamic threshold methods to extract fatigue characteristics and (3) a stable fatigue measurement based on fusing percentage of eyelid closure (PERCLOS) and proportion of long closure duration blink (PLCDB). The excellent performance of our proposed algorithm and methods are verified in experiments. The experimental results show that our key point alignment algorithm is robust to different scenes, and the performance of our proposed fatigue measurement is more reliable due to the fusion of PERCLOS and PLCDB.

Smart Driver Fatigue Detection System Using CNN

Abstract: Road accidents frequently result from fatigued or sleepy drivers. More people die and are killed worldwide each year. Driver fatigue is a major factor in many accidents. It is currently one of the major causes of accidents. Numerous facial expressions, such as frequent yawning, frequent eye blinking, and head positioning, might indicate how tired someone is. Computer vision is the most suitable and practical technology to solve this issue since it uses these sensory properties. The suggested methodology employs Dlib, CNN, and OpenCV. Dlib estimates the positions of 68 coordinates (x,y) that help in mapping the facial points on a person’s face. The 2904 pictures that make up the collections classified are Open Eyes, Closed Eyes, Yawning,or No-Yawning. A convolutional neural network (CNN) will be used to determine the mouth and eye states from the Regionof Interest (ROI) photographs. The amount of mouth opening (FOM) and the percentage of eyelid closure (PERCLOS) overthe pupil are the measures used to measure weariness over time. The recommended system determines whether the driver is sleepyand sounds a warning when it detects their eyes and yawns fora certain amount of time. The accuracy of the model has been discussed.

Fatigue Detection Algorithm Based on Eye Multifeature Fusion

Fatigue driving is a critical cause of traffic accidents. Different from a single feature-based fatigue detection algorithm, this article uses the Dlib toolkit to mark facial feature points and presents a novel fatigue detection algorithm considering the eye’s multifeature and giving an optimal weight distribution for this fusion model, which intends not only to improve the reliability and error tolerance rate of detection but also to address the issue that traditional algorithms cannot make a balance between real time and accuracy at the same time. The eye’s multifeature fusion algorithm simultaneously detects the eye aspect ratio (EAR), the percentage of eyelid closure over the pupil (PERCLOS), the blink frequency (BF), and the pupil occlusion rate (POR). Afterward, the results are normalized to judge the driver’s fatigue level and give a corresponding warning. The testing results show that the accuracy of our proposed algorithm reaches 95%, exhibiting a solid potential in fatigue detection applications.

Improved Driver Fatigue Detection using Blinking and Yawning Rate

Fatigue-induced driving is a prevalent cause of road accidents. Hence, multiple authors have proposed various methods to detect driver drowsiness at an early stage to prevent such occurrences. This scholarly piece introduces an efficient approach to detect exhaustion by examining the blinking and yawning rate of driver. The partitioning process generates a representation, which assists in identifying the pupil and mouth. To ensure greater accuracy and applicability in determining eye openness, a composite network comprising a classification network and an evaluation network is developed. The partitioning network employs a lightweight Linknet architecture to categorize pixels in eye and mouth images, which enables the accurate extraction of the pupil and mouth. The decision network utilizes the representation extracted to assess the extent of eye and mouth openness. Finally, the model is tested on a larger dataset, and it achieves state-of- the-art accuracy of 92%. KEYWORDS: LinkNet, fatigue detection, facial features, PERCLOS(percentage of eyelid closure ),EOR(Eye opening ratio), MOR(mouth opening ratio)

Fatigue Working Detection Based on Facial Multifeature Fusion

Working under fatigue states is not only inefficient but also brings a series of safety concerns and health problems. This article presents a novel fatigue detection algorithm based on facial multifeature fusion, exhibiting promising properties of immediacy and accuracy. Following the video processing of marking the gray image frames and the histogram equalization using the Dlib toolkit, the facial features are extracted based on the facial marker points and then evaluated to obtain the eye aspect ratio (EAR), mouth aspect ratio (MAR), and head Euler angles (HEAs) in real-time. These evaluation indexes can further contribute to calculating blinking frequency (BF), percentage of eyelid closure (PERCLOS) over the pupil over time, yawning frequency (YF), and nodding frequency (NF), of which the four parameters are normalized to establish the detection model, showing the capability of identifying the fatigue grade with high accuracy and quick response. The actual test verifies the algorithm’s reliability, and the results show that the accuracy of detecting the fatigue behaviors reaches more than 94.4%, and the final judgment perfectly matches the actual physiological state based on ensuring the real-time performance of the detection.

On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features.

Fatigue detection for air traffic controllers is an important yet challenging problem in aviation safety research. Most of the existing methods for this problem are based on facial features. In this paper, we propose an ensemble learning model that combines both facial features and voice features and design a fatigue detection method through multifeature fusion, referred to as Facial and Voice Stacking (FV-Stacking). Specifically, for facial features, we first use OpenCV and Dlib libraries to extract mouth and eye areas and then employ a combination of M-Convolutional Neural Network (M-CNN) and E-Convolutional Neural Network (E-CNN) to determine the state of mouth and eye closure based on five features, i.e., blinking times, average blinking time, average blinking interval, Percentage of Eyelid Closure over the Pupil over Time (PERCLOS), and Frequency of Open Mouth (FOM). For voice features, we extract the Mel-Frequency Cepstral Coefficients (MFCC) features of speech. Such facial features and voice features are fused through a carefully designed stacking model for fatigue detection. Real-life experiments are conducted on 14 air traffic controllers in Southwest Air Traffic Management Bureau of Civil Aviation of China. The results show that the proposed FV-Stacking method achieves a detection accuracy of 97%, while the best accuracy achieved by a single model is 92% and the best accuracy achieved by the state-of-the-art detection methods is 88%.

Drowsiness Detection System Based on Machine Learning Using Eye State

Drowsiness is one of the major causes of driver-induced traffic accidents. The interactive systems developed to reduce road accidents by alerting drivers is called as Advanced Driver Assistance Systems (ADAS). The most important ADAS are Lane Departure Warning System, Front Collision Warning System and Driver Drowsiness Systems. In this study, an ADAS system based on eye state detection is presented to detect driver drowsiness. First, Viola-Jones algorithm approach is used to detect the face and eye areas in the proposed method. The detected eye region is classified as closed or open by making use of a machine learning method. Finally, the eye conditions are analyzed at time domain with PERcentage of eyelid CLOsure (PERCLOS) metric and drowsiness conditions are determined by Support Vector Machine (SVM), kNN and decision tree classifiers. The proposed methods tested on 7 real people and drowsiness states are detected at 99.77%, 94.35%, and 96.62% accuracy, respectively.

Drowsiness Detection System Based on PERCLOS and Facial Physiological Signal.

Accidents caused by fatigue occur frequently, and numerous scholars have devoted tremendous efforts to investigate methods to reduce accidents caused by fatigued driving. Accordingly, the assessment of the spirit status of the driver through the eyes blinking frequency and the measurement of physiological signals have emerged as effective methods. In this study, a drowsiness detection system is proposed to combine the detection of LF/HF ratio from heart rate variability (HRV) of photoplethysmographic imaging (PPGI) and percentage of eyelid closure over the pupil over time (PERCLOS), and to utilize the advantages of both methods to improve the accuracy and robustness of drowsiness detection. The proposed algorithm performs three functions, including LF/HF ratio from HRV status judgment, eye state detection, and drowsiness judgment. In addition, this study utilized a near-infrared webcam to obtain a facial image to achieve non-contact measurement, alleviate the inconvenience of using a contact wearable device, and for use in a dark environment. Furthermore, we selected the appropriate RGB channel under different light sources to obtain LF/HF ratio from HRV of PPGI. The main drowsiness judgment basis of the proposed drowsiness detection system is the use of algorithm to obtain sympathetic/parasympathetic nervous balance index and percentage of eyelid closure. In the experiment, there are 10 awake samples and 30 sleepy samples. The sensitivity is 88.9%, the specificity is 93.5%, the positive predictive value is 80%, and the system accuracy is 92.5%. In addition, an electroencephalography signal was used as a contrast to validate the reliability of the proposed method.

Train Driver Fatigue Detection Using Eye Feature Vector and Support Vector Machine

Fatigue driving is one of the main causes of traffic accidents. The eye features are the important cues of fatigue detection. In order to improve the accuracy and robustness of detection based on a single eye feature, we propose a fatigue detection algorithm based on the eye feature (EFV) vector. Firstly, the coordinates of the eye region were localized with facial landmarks detector and the landmarks geometric relation (LGR) was calculated as a feature value. Secondly, a deep transfer learning network was designed to classify the driver eye state on a small dataset. The probability value of the eyes being open state was calculated. Then an eye feature vector was constructed to overcome the limitations of a single fixed threshold and a support vector machine (SVM) model was trained for eye state classification recognition. Finally, the performance of the proposed detection model was evaluated by the percentage of eyelid closure over time (PERCLOS) criterion. The results show that the accuracy of this model can reach 91.67% on the test database, which is higher than the single-feature-based method. This work lays a foundation for the online fatigue detection of train drivers and the deployment of the train driving monitoring system.

Driver Drowsiness Detection Using Gray Wolf Optimizer Based on Face and Eye Tracking

It is critical today to provide safe and collision-free transport. As a result, identifying the driver’s drowsiness before their capacity to drive is jeopardized. An automated hybrid drowsiness classification method that incorporates the artificial neural network (ANN) and the gray wolf optimizer (GWO) is presented to discriminate human drowsiness and fatigue for this aim. The proposed method is evaluated in alert and sleep-deprived settings on the driver drowsiness detection of video dataset from the National Tsing Hua University Computer Vision Lab. The video was subjected to various video and image processing techniques to detect the drivers’ eye condition. Four features of the eye were extracted to determine the condition of drowsiness, the percentage of eyelid closure (PERCLOS), blink frequency, maximum closure duration of the eyes, and eye aspect ratio (ARE). These parameters were then integrated into an ANN and combined with the proposed method (gray wolf optimizer with ANN [GWOANN]) for drowsiness classification. The accuracy of these models was calculated, and the results demonstrate that the proposed method is the best. An Adadelta optimizer with 3 and 4 hidden layer networks of (13, 9, 7, and 5) and (200, 150, 100, 50, and 25) neurons was utilized. The GWOANN technique had 91.18% and 97.06% accuracy, whereas the ANN model had 82.35% and 86.76%.

Driver drowsiness estimation using EEG signals with a dynamical encoder\u2013decoder modeling framework

Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.

A Novel Fatigue Driving State Recognition and Warning Method Based on EEG and EOG Signals.

Traffic accidents are easily caused by tired driving. If the fatigue state of the driver can be identified in time and a corresponding early warning can be provided, then the occurrence of traffic accidents could be avoided to a large extent. At present, the recognition of fatigue driving states is mostly based on recognition accuracy. Fatigue state is currently recognized by combining different features, such as facial expressions, electroencephalogram (EEG) signals, yawning, and the percentage of eyelid closure over the pupil over time (PERCLoS). The combination of these features increases the recognition time and lacks real-time performance. In addition, some features will increase error in the recognition result, such as yawning frequently with the onset of a cold or frequent blinking with dry eyes. On the premise of ensuring the recognition accuracy and improving the realistic feasibility and real-time recognition performance of fatigue driving states, a fast support vector machine (FSVM) algorithm based on EEGs and electrooculograms (EOGs) is proposed to recognize fatigue driving states. First, the collected EEG and EOG modal data are preprocessed. Second, multiple features are extracted from the preprocessed EEGs and EOGs. Finally, FSVM is used to classify and recognize the data features to obtain the recognition result of the fatigue state. Based on the recognition results, this paper designs a fatigue driving early warning system based on Internet of Things (IoT) technology. When the driver shows symptoms of fatigue, the system not only sends a warning signal to the driver but also informs other nearby vehicles using this system through IoT technology and manages the operation background.

Driver Fatigue Detection Based on Residual Channel Attention Network and Head Pose Estimation

Driver fatigue is the culprit of most traffic accidents. Visual technology can intuitively judge whether the driver is in the state of fatigue. A driver fatigue detection system based on the residual channel attention network (RCAN) and head pose estimation is proposed. In the proposed system, Retinaface is employed for face location and outputs five face landmarks. Then the RCAN is proposed to classify the state of eyes and the mouth. The RCAN includes a channel attention module, which can adaptively extract key feature vectors from the feature map, which significantly improves the classification accuracy of the RCAN. In the self-built dataset, the classification accuracy of the eye state of the RCAN reaches 98.962% and that of the mouth state reaches 98.561%, exceeding other classical convolutional neural networks. The percentage of eyelid closure over the pupil over time (PERCLOS) and the mouth opening degree (POM) are used for fatigue detection based on the state of eyes and the mouth. In addition, this article proposes to use a Perspective-n-Point (PnP) method to estimate the head pose as an essential supplement for driving fatigue detection and proposes over-angle to evaluate whether the head pose is excessively deflected. On the whole, the proposed driver fatigue system integrates 3D head pose estimation and fatigue detection based on deep learning. This system is evaluated by the four datasets and shows success of the proposed method with their high performance.

Driver Fatigue Detection Based On Facial Feature Analysis

Fatigue driving is one of the main causes of traffic accidents. In recent years, considerable attention has been paid to fatigue detection systems, which is an important solution for preventing fatigue driving. In order to prevent and reduce fatigue driving, a driver fatigue detection system based on computer vision is proposed. In this system, an improved face detection method is used to detect the driver’s face from the image obtained by a charge coupled device (CCD) camera. Then, the feature points of the eyes and mouth are located by an ensemble of regression trees. Next, fatigue characteristic parameters are calculated by the improved percentage of eyelid closure over the pupil over time algorithm. Finally, the state of drivers is evaluated by using a fuzzy neural network. The system can effectively monitor and remind the state of drivers so as to significantly avoid or decrease the occurrence of traffic accidents. The experimental results show that the system is of wonderful real-time performance and accurate recognition rate, so it meets the requirements of practicality in driver fatigue detection greatly.

Predicting Driver Fatigue in Monotonous Automated Driving with Explanation using GPBoost and SHAP

ABSTRACT Research indicates that monotonous automated driving increases the incidence of fatigued driving. Although many prediction models based on advanced machine learning techniques were proposed to monitor driver fatigue, especially in manual driving, little is known about how these black-box machine learning models work. In this paper, we proposed a combination of Gaussian Process Boosting (GPBoost) and SHapley Additive exPlanations (SHAP) to predict driver fatigue with explanations. First, in order to obtain the ground truth of driver fatigue, we used PERCLOS (percentage of eyelid closure over the pupil over time) between 0 and 100 as the response variable. Second, we built a driver fatigue regression model using both physiological and behavioral measures with GPBoost that was able to address the within-subjects correlations. This model outperformed other selected machine learning models with root-mean-squared error (RMSE) = 2.965, mean absolute error (MAE) = 1.407, and adjusted . Third, we employed SHAP to identify the most important predictor variables and uncovered the black-box GPBoost model by showing the main effects of the most important predictor variables globally and explaining individual predictions locally. Such an explainable driver fatigue prediction model offered insights into how to intervene in automated driving when necessary, such as during the takeover transition period from automated driving to manual driving.

Real-time detection method of driver fatigue state based on deep learning of face video

The use of face video information for driver fatigue detection has received extensive attention because of its low cost and non-invasiveness. However, the current vehicle-mounted embedded device has insufficient memory and limited computing power, which cannot complete the real-time detection of driver fatigue based on deep learning. Therefore, this paper designs a lightweight neural network model to solve this problem. The model includes object detection and fatigue detection. First, a lightweight object detection network is designed, which can quickly identify the opening and closing states of the driver’s eyes and mouth in the time series video. Secondly, the EYE-MOUTH (EM) driver fatigue detection model is designed, which encodes the driver’s eye and mouth opening and closing state, and calculates the driver’s PERCLOS (Percentage of Eyelid Closure over the Pupil) and FOM (Frequency of Open Mouth) according to the coding sequence. Finally, the multi-feature fusion judgment algorithm is used to realize the judgment of the driver’s fatigue state. The experimental results show that our method has an accuracy rate of 98.30% for drowsiness and yawning behaviors in a real vehicle environment, and a detection speed of 27FPS, which is better than other advanced methods and meets the requirements of real-time detection.

Human Fatigue Evaluation During Smart Working Sessions

AbstractIn the entire world, 2020 will be remembered as a pandemic year, in which COVID‐19 virus has forced to lockdown entire cities and countries. In this context, the concept of work is radically changed, especially for production companies that are forced to temporarily close their production plants. On the other hand, other professional figures, such as engineers, analysts, call center workers, school and university professors, etc., are experimenting a different way of working, the so‐called “smart working”, namely a type of work free of time and place constraints. This paper aims at investigating how the change of working conditions affects the perceived fatigue by workers. In particular, an experimental study is conducted on some smart workers by means of the PERCLOS (PERcentage of eyelid CLOSure) method, that allows to evaluate sleepiness by recording workers faces during the job and analyzing their mouths and eyes.

Video-based sleep detection using ocular signals under the standard conditions of the maintenance of wakefulness test in patients with sleep disorders

Objective. Excessive sleepiness is a physiological reaction to sleep deficiency but can also be caused by underlying medical conditions. Detecting sleep is essential in preventing accidents and for medical diagnostics. Polysomnography (PSG) is considered the gold standard for the detection of sleep. More convenient video-based methods for detecting sleepiness have recently emerged. Approach. The possibility of detecting sleep using video-based ocular signals will be assessed using PSG for reference. Ocular signals and EEG are recorded in parallel under the conditions of the maintenance of wakefulness test (MWT) in 30 patients with sleep disorders. Main results. In detecting sleep, the ocular signal percentage of eyelid closure (PERCLOS) is superior to other ocular signals, resulting in an area under the curve of 0.88. Using a PERCLOS cutoff value of 0.76, sleep is correctly detected with a sensitivity of 89%, a specificity of 76%, the sleep latency is moderately correlated to the reference (rho = 0.66, p < 0.05) and the 95% confidence interval is ±21.1 min. Significance. Ocular signals can facilitate the detection of sleep under the conditions of the MWT but sleep detection should not solely rely on ocular signals. If PSG recordings are not practicable or if a signal is needed that responds relatively early in the wake/sleep transition, the use of PERCLOS for the detection of sleep is reasonable.