Blink Detection Research Articles

Noise robustness evaluation of image processing algorithms for eye blinking detection

Robust algorithms for eye blinking detection are required due to the effects of noisy environments and varying light conditions on image-based detection methods. This paper compares five non-supervised image-based algorithms for eye blinking detection, evaluating their robustness to additive Gaussian noise. The algorithms were tested on a video dataset acquired using a smartphone and an ophthalmology chin rest. Through Monte Carlo simulation that introduces Gaussian noise at different intensities, we evaluate the algorithms’ precision, sensitivity, and F1-scores for frame classification, True Positive Rate (TPR), False Discovery Rate (FDR) and F1-score for the event detector. The results of experimental tests reveal significant variations in algorithms’ performance with increasing noise levels. Notably, the Image Correlation (IC) algorithm demonstrates superior eye blinking detection capabilities under various noise conditions, emerging as the most robust algorithm among those tested. This distinction highlights the potential of IC for reliable blink detection in noisy environments.

Multi‐parameter fusion driver fatigue detection method based on facial fatigue features

AbstractFatigued driving is one of the main causes of traffic accidents. In order to improve the detection speed of fatigue driving recognition, this paper proposes a driver fatigue detection method based on multi‐parameter fusion of facial features. It uses a cascaded Adaboost object classifier to detect faces in video streams. The DliB library is employed for facial key point detection, which locates the driver's eyes and mouth to determine their states. The eye aspect ratio (EAR) is calculated to detect eye closure, and the mouth aspect ratio (MAR) is calculated to detect yawning frequency and count. The detected percentage of eye closure (PERCLOS) value is combined with yawning frequency and count, and a multi‐feature fusion approach is used for fatigue detection. Experimental results show that the accuracy of blink detection is 91% and the accuracy of yawn detection is 96.43%. Furthermore, compared to the models mentioned in the comparative experiments, this model achieves two to four times faster detection times while maintaining accuracy.

Using Pupil Diameter for Psychological Resilience Assessment in Medical Students Based on SVM and SHAP Model.

Effectively assessing psychological resilience for medical students is vital for identifying at-risk individuals and developing tailored interventions. At present, few studies have combined physiological indexes of the human body and machine learning for psychological resilience assessment. This study presents a novel approach that employs pupil diameter features and machine learning to predict psychological resilience risk objectively. Firstly, we designed a stimulus paradigm (via auditory and visual stimuli) and collected pupil diameter data from participants using eye-tracking technology. Secondly, the pupil data was preprocessed, including linear interpolation, blink detection, and subtractive baseline correction. Thirdly, statistical metrics were extracted and optimal feature subsets were obtained by Recursive Feature Elimination with Cross-Validation (RFECV). Subsequently, the classification models, including Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost), were trained. The experimental results show that the SVM model has the best performance, and its balance accuracy, recall, and AUC reach 0.906, 0.89, and 0.932, respectively. Finally, we leveraged the Shapley additive explanation (SHAP) model for interpretability analysis. It revealed auditory stimuli have a more significant effect than visual stimuli in psychological resilience assessment. These findings suggested that pupil diameter could be a vital metric for assessing psychological resilience.

MEBAL2 database and benchmark: Image-based multispectral eyeblink detection

This work introduces a new multispectral database and framework to train and evaluate eyeblink detection in RGB and Near-Infrared (NIR). Our contributed dataset (mEBAL2, multimodal EyeBlink and Attention Level estimation, Version 2) is the largest existing eyeblink database, representing a great opportunity to improve data-driven multispectral approaches for blink detection and related applications (e.g., attention level estimation). mEBAL2 includes 21,100 image sequences from 180 different students (more than 2 million labeled images in total) while conducting a number of e-learning tasks of varying difficulty or taking a real course on HTML initiation through the edX MOOC platform. mEBAL2 uses multiple sensors, including two Near-Infrared (NIR) and one RGB camera to capture facial gestures during the execution of the tasks, as well as an Electroencephalogram (EEG) band to get the cognitive activity of the user and blinking events. Furthermore, this work proposes 3 data-driven approaches as benchmarks for blink detection on mEBAL2, where the architecture based on Convolutional Long Short-Term Memory (ConvLSTM) achieved performances of up to 99%. The experiments explored whether combining RGB and NIR spectrum data improves blink detection in training and architectures that merge both types of data. Experiments showed that the NIR spectrum enhances results, even when only RGB images are available during inference. Finally, the generalization capacity of the proposed eyeblink detectors, along with state-of-the-art eyeblink detection implementations, is validated in wilder and more challenging environments like the HUST-LEBW dataset to show the usefulness of mEBAL2 to train a new generation of data-driven approaches for eyeblink detection.

Feature Extraction from Regenerated EEG – A Better Approach for ICA Based Eye Blink Artifact Detection

Purpose: Independent Component Analysis (ICA) decomposition is a commonly used technique for eye blink artifact detection from Electroencephalogram (EEG) signals. Feature extraction from the decomposed ICs is a prime step for blink detection. This paper presents a new model of eye blink detection for ICA based approach, where the decomposed ICs are projected to their corresponding EEG segments (ReEEG), and feature extraction is performed on the ReEEG instead of the IC. ReEEG represents the eye blink activity more distinctly. Hence, ReEEG-based feature extraction is more potential in detecting eye blink artifacts than the traditional IC-based feature extraction.
 Materials and Methods: This paper employs twelve EEG features to substantiate the superiority of ReEEG over IC. Support Vector Machine (SVM) is used as a classifier. A dataset, having 2638 clinical EEG epochs, is employed. All the considered twelve features are extracted from ReEEG and fed to SVM one at a time for blink detection. Then the obtained results are compared with an IC-based model with the same features.
 Results: The comparison reveals the success of the proposed ReEEG-based blink detection approach over the traditional IC-based approach. Accuracy, precision, recall, and f1 scores are calculated as performance measuring metrics. For almost all features, ReEEG-based approach achieved up to 12.25% higher accuracy, 24.95% higher precision, 13.49% higher recall, and 12.89% higher f1 score than the IC-based traditional method.
 Conclusion: The proposed model will be useful for researchers in dealing with the eye blink artifacts of EEG signals with more efficacy.

Eye-LRCN: A Long-Term Recurrent Convolutional Network for Eye Blink Completeness Detection.

Computer vision syndrome causes vision problems and discomfort mainly due to dry eye. Several studies show that dry eye in computer users is caused by a reduction in the blink rate and an increase in the prevalence of incomplete blinks. In this context, this article introduces Eye-LRCN, a new eye blink detection method that also evaluates the completeness of the blink. The method is based on a long-term recurrent convolutional network (LRCN), which combines a convolutional neural network (CNN) for feature extraction with a bidirectional recurrent neural network that performs sequence learning and classifies the blinks. A Siamese architecture is used during CNN training to overcome the high-class imbalance present in blink detection and the limited amount of data available to train blink detection models. The method was evaluated on three different tasks: blink detection, blink completeness detection, and eye state detection. We report superior performance to the state-of-the-art methods in blink detection and blink completeness detection, and remarkable results in eye state detection.

Design recommendations for voluntary blink interactions based on pressure sensors

Voluntary blink control is vital in eye-controlled human–computer interaction (ECHCI). Pressure sensors, known for their low power consumption and strong resistance to interference, are suitable for blink detection. Existing research has predominantly focused on sensor materials and signal processing, and studies on the characteristics and optimized design of voluntary blink actions in ECHCI remain relatively limited. Therefore, this study presents a blink detection system based on pressure sensors and investigates the efficiency of voluntary blink actions. We examined the effects of two independent variables, blink count (single, double, and triple) and blink side (bilateral and unilateral), on recognition accuracy and three time-related variables. Subsequently, the subjective workload and evaluation experiments were conducted for each blink action. The results indicate that the blink count has a more pronounced impact on interaction efficiency and comfort, with continuous blink actions exceeding three blinks not being recommended. Bilateral blinks were perceived as more natural and comfortable than unilateral blinks. Moreover, we discovered that the participants exhibited greater attentiveness and focus when performing double bilateral blink actions, making it suitable for important commands. Based on the experimental results, the top three recommended voluntary blink actions are single bilateral blink (SB), double bilateral blink (DB), and single unilateral blink (SU). Finally, a test involving controlling the movement of a toy car using the recommended blink actions validates the usability of the actions and detection system. These findings provide valuable guidance for the design and optimization of voluntary-blink interaction systems, further enhancing this interaction method’s feasibility.

What is a blink? Classifying and characterizing blinks in eye openness signals

Blinks, the closing and opening of the eyelids, are used in a wide array of fields where human function and behavior are studied. In data from video-based eye trackers, blink rate and duration are often estimated from the pupil-size signal. However, blinks and their parameters can be estimated only indirectly from this signal, since it does not explicitly contain information about the eyelid position. We ask whether blinks detected from an eye openness signal that estimates the distance between the eyelids (EO blinks) are comparable to blinks detected with a traditional algorithm using the pupil-size signal (PS blinks) and how robust blink detection is when data quality is low. In terms of rate, there was an almost-perfect overlap between EO and PS blink (F1 score: 0.98) when the head was in the center of the eye tracker’s tracking range where data quality was high and a high overlap (F1 score 0.94) when the head was at the edge of the tracking range where data quality was worse. When there was a difference in blink rate between EO and PS blinks, it was mainly due to data loss in the pupil-size signal. Blink durations were about 60 ms longer in EO blinks compared to PS blinks. Moreover, the dynamics of EO blinks was similar to results from previous literature. We conclude that the eye openness signal together with our proposed blink detection algorithm provides an advantageous method to detect and describe blinks in greater detail.

Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation.

Eye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a potential lack of awareness regarding these tests and the absence of appropriate tools for detecting nystagmus. This study aimed to develop a nystagmus measurement algorithm using a lightweight deep-learning model that recognizes the ocular regions. The deep learning model was used to segment the eye regions, detect blinking, and determine the pupil center. The model was trained using images extracted from video clips of a clinical battery of eye movement tests and synthesized images reproducing real eye movement scenarios using virtual reality. Each eye image was annotated with segmentation masks of the sclera, iris, and pupil, with gaze vectors of the pupil center for eye tracking. We conducted a comprehensive evaluation of model performance and its execution speeds in comparison to various alternative models using metrics that are suitable for the tasks. The mean Intersection over Union values of the segmentation model ranged from 0.90 to 0.97 for different classes (sclera, iris, and pupil) across types of images (synthetic vs. real-world images). Additionally, the mean absolute error for eye tracking was 0.595 for real-world data and the F1 score for blink detection was ≥ 0.95, which indicates our model is performing at a very high level of accuracy. Execution speed was also the most rapid for ocular object segmentation under the same hardware condition as compared to alternative models. The prediction for horizontal and vertical nystagmus in real eye movement video revealed high accuracy with a strong correlation between the observed and predicted values (r = 0.9949 for horizontal and r = 0.9950 for vertical; both p < 0.05). The potential of our model, which can automatically segment ocular regions and track nystagmus in real time from eye movement videos, holds significant promise for emergency settings or remote intervention within the field of neurotology.

Eye Blink Detection- A New System for Driver Drowsiness and Distraction Detection

The advancement of technology has led to innovative solutions for ensuring road safety, particularly in the detection of driver drowsiness and distraction. This abstract presents a novel approach using eye blink detection as a fundamental component in a system designed to mitigate the risks associated with drowsy and distracted driving. The proposed system integrates cutting-edge computer vision techniques to accurately detect and analyze eye blinks in real-time. By utilizing high-resolution cameras installed within the vehicle, the system captures and processes images of the driver's face, focusing on the eyes to track blinking patterns and durations. Through sophisticated algorithms and machine learning models, it distinguishes between normal blinks and extended periods of eye closure, indicative of drowsiness or distraction. Furthermore, the system incorporates contextual awareness by considering various factors such as environmental conditions, driving behavior, and facial expressions to enhance the accuracy of detection. Machine learning algorithms trained on diverse datasets enable the system to adapt and improve its performance over time, making it more adept at recognizing subtle variations in blink patterns unique to each driver. The applications of this system are multifaceted. Firstly, it serves as an early warning system, alerting drivers when signs of drowsiness or distraction are detected. This proactive approach aims to prevent accidents by prompting drivers to take necessary breaks or refocus their attention on the road. Moreover, the system can integrate with existing driver assistance technologies, allowing for automated interventions, such as adjusting seat vibrations or emitting audible alerts to re-engage the driver. In conclusion, the proposed eye blink detection system represents a significant advancement in the realm of driver safety technologies. By leveraging sophisticated algorithms and real-time analysis of blink patterns, it offers a promising solution to mitigate the risks associated with driver drowsiness and distraction, ultimately contributing to safer roads and reducing the likelihood of accidents. KEYWORDS: Eye blinking, Detection, Driver, Drowsiness, Distraction, Detection, computer vision, machine learning, Deep learning, image processing.

A Wearable Asynchronous Brain-Computer Interface Based on EEG-EOG Signals With Fewer Channels.

Brain-computer interfaces (BCIs) have tremendous application potential in communication, mechatronic control and rehabilitation. However, existing BCI systems are bulky, expensive and require laborious preparation before use. This study proposes a practical and user-friendly BCI system without compromising performance. A hybrid asynchronous BCI system was developed based on an elaborately designed wearable electroencephalography (EEG) amplifier that is compact, easy to use and offers a high signal-to-noise ratio (SNR). The wearable BCI system can detect P300 signals by processing EEG signals from three channels and operates asynchronously by integrating blink detection. The wearable EEG amplifier obtains high quality EEG signals and introduces preprocessing capabilities to BCI systems. The wearable BCI system achieves an average accuracy of 94.03±4.65%, an average information transfer rate (ITR) of 31.42±7.39 bits/min and an average false-positive rate (FPR) of 1.78%. The experimental results demonstrate the feasibility and practicality of the developed wearable EEG amplifier and BCI system. Wearable asynchronous BCI systems with fewer channels are possible, indicating that BCI applications can be transferred from the laboratory to real-world scenarios.

Application Specific Reconfigurable Processor for Eyeblink Detection from Dual-Channel EOG Signal

The electrooculogram (EOG) is one of the most significant signals carrying eye movement information, such as blinks and saccades. There are many human–computer interface (HCI) applications based on eye blinks. For example, the detection of eye blinks can be useful for paralyzed people in controlling wheelchairs. Eye blink features from EOG signals can be useful in drowsiness detection. In some applications of electroencephalograms (EEGs), eye blinks are considered noise. The accurate detection of eye blinks can help achieve denoised EEG signals. In this paper, we aimed to design an application-specific reconfigurable binary EOG signal processor to classify blinks and saccades. This work used dual-channel EOG signals containing horizontal and vertical EOG signals. At first, the EOG signals were preprocessed, and then, by extracting only two features, the root mean square (RMS) and standard deviation (STD), blink and saccades were classified. In the classification stage, 97.5% accuracy was obtained using a support vector machine (SVM) at the simulation level. Further, we implemented the system on Xilinx Zynq-7000 FPGAs by hardware/software co-design. The processing was entirely carried out using a hybrid serial–parallel technique for low-power hardware optimization. The overall hardware accuracy for detecting blinks was 95%. The on-chip power consumption for this design was 0.8 watts, whereas the dynamic power was 0.684 watts (86%), and the static power was 0.116 watts (14%).

On validating a generic camera‐based blink detection system for cognitive load assessment

AbstractDetecting the human operator's cognitive state is paramount in settings wherein maintaining optimal workload is necessary for task performance. Blink rate is an established metric of cognitive load, with a higher blink frequency being observed under conditions of greater workload. Measuring blink rate requires the use of eye‐trackers which limits the adoption of this metric in the real‐world. The authors aim to investigate the effectiveness of using a generic camera‐based system as a way to assess the user's cognitive load during a computer task. Participants completed a mental task while sitting in front of a computer. Blink rate was recorded via both the generic camera‐based system and a scientific‐grade eye‐tracker for validation purposes. Cognitive load was also assessed through the performance in a single stimulus detection task. The blink rate recorded via the generic camera‐based approach did not differ from the one obtained through the eye‐tracker. No meaningful changes in blink rate were however observed with increasing cognitive load. Results show the generic‐camera based system may represent a more affordable, ubiquitous means for assessing cognitive workload during computer task. Future work should further investigate ways to increase its accuracy during the completion of more realistic tasks.

Real-time eye blink detection using general cameras: a facial landmarks approach

Eyes are essential in Human-Computer Interaction (HCI) as they provide valuable insights into a person's thoughts and intentions. However, current eye movement analysis methods require specialized equipment or high-quality videos, making them less accessible and usable. This paper proposes a real-time eye blink detection algorithm that uses standard cameras, making it widely applicable and convenient. The approach achieves accurate and efficient eye blink detection by leveraging the Eye Aspect Ratio (EAR) algorithm and facial landmarks technique. This paper developed a Python application and conducted tests using a laptop webcam to validate the performance and practicality of the algorithm across various settings. The algorithm's effectiveness depends on carefully tuning parameters such as threshold and frame rate. The results demonstrate the algorithm's potential in real-time eye blink detection, with potential applications in drowsiness detection during driving, prevention of Computer Vision Syndrome, and assistance for individuals with disabilities. By enabling eye blink detection using commonly available cameras, the algorithm paves the way for integrating eye movement analysis into everyday devices and systems, enhancing user experience and enabling more natural interactions. Further refinement and optimization of this approach hold promise for a wide range of applications in HCI, healthcare, and beyond, opening up new possibilities for research and innovation.

BlinkLinMulT: Transformer-Based Eye Blink Detection.

This work presents BlinkLinMulT, a transformer-based framework for eye blink detection. While most existing approaches rely on frame-wise eye state classification, recent advancements in transformer-based sequence models have not been explored in the blink detection literature. Our approach effectively combines low- and high-level feature sequences with linear complexity cross-modal attention mechanisms and addresses challenges such as lighting changes and a wide range of head poses. Our work is the first to leverage the transformer architecture for blink presence detection and eye state recognition while successfully implementing an efficient fusion of input features. In our experiments, we utilized several publicly available benchmark datasets (CEW, ZJU, MRL Eye, RT-BENE, EyeBlink8, Researcher's Night, and TalkingFace) to extensively show the state-of-the-art performance and generalization capability of our trained model. We hope the proposed method can serve as a new baseline for further research.

The Assistance of Eye Blink Detection for Two- Factor Authentication

This paper discusses the implementation of a blink detection method using 68 facial markers and the eye aspect ratio (EAR) to provide strong protection for access systems. It investigates the importance of 68 facial markers and explores how to use eye landmarks to calculate the eye aspect ratio. Access systems, which should have good security measures and be difficult to decipher, are typically safeguarded by passwords or multi-factor verification, such as passwords combined with facial recognition. However, these methods have inherent weaknesses, including the risk of shoulder surfing with passwords and the potential to be deceived by fake face images with facial recognition. To address these issues, a two-factor authentication method by using password with eye blink recognition is proposed as an effective solution for access control. By incorporating real-time eye blinking action, the system can avoid the use of fake images and prevent shoulder spoofing. To demonstrate the practical application of eye blink detection for enhanced two-factor authentication, a web application called "Eblink" is introduced. Functional tests have been conducted to validate the application's core features.

A Real-Time Eye Tracking System for the Detection of Eye Blinks.

This paper proposes an eye blink detection system that automatically detects eye blinks, which can be an indicator of fatigue or cognitive load, among others. As a key feature, the real-time capability of the system is being required to use it, for example, as a monitoring system for people in potentially critical situations (e.g., drivers or operators of heavy machinery). The system uses the Viola-Jones algorithm for face detection and the median flow tracker to track the face in video sequences. Eye detection is implemented using face proportions, and template matching is used for blink detection. The resulting system processes 40-47 frames per second on default consumer hardware and achieves an accuracy of 80.33% and a precision of 85.22% in the evaluation. The proposed system shows promising results under ideal viewing conditions but has difficulty maintaining high precision during head movements. The proposed system could be integrated with various health-related assistance systems to monitor the individual's well-being in real time, as long as their head is observed from the front if possible.

On Validating a Generic Video-Based Blink Detection System for Cognitive Load Detection

This work aims to validate using a video-based blink detection system for detecting changes in cognitive load. Participants completed a cognitive task with increasing levels of difficulty. Blink rate was recorded via both our video-based system and a scientific-grade eye-tracker. Results showed no differences in the blink rates recorded by the two systems. However, while strong evidence was found that the blink rate recorded through the eye-tracker increased under greater cognitive task demand, no such pattern was observed for the output from the video-based system. Our findings show that the overall performance of the video-based system was comparable with that of the eye-tracker's, but its sensitivity in detecting changes in cognitive load was inferior to that of its counterpart's.

Deep learning models for webcam eye tracking in online experiments

Eye tracking is prevalent in scientific and commercial applications. Recent computer vision and deep learning methods enable eye tracking with off-the-shelf webcams and reduce dependence on expensive, restrictive hardware. However, such deep learning methods have not yet been applied and evaluated for remote, online psychological experiments. In this study, we tackle critical challenges faced in remote eye tracking setups and systematically evaluate appearance-based deep learning methods of gaze tracking and blink detection. From their own homes and laptops, 65 participants performed a battery of eye tracking tasks including (i) fixation, (ii) zone classification, (iii) free viewing, (iv) smooth pursuit, and (v) blink detection. Webcam recordings of the participants performing these tasks were processed offline through appearance-based models of gaze and blink detection. The task battery required different eye movements that characterized gaze and blink prediction accuracy over a comprehensive list of measures. We find the best gaze accuracy to be 2.4° and precision of 0.47°, which outperforms previous online eye tracking studies and reduces the gap between laboratory-based and online eye tracking performance. We release the experiment template, recorded data, and analysis code with the motivation to escalate affordable, accessible, and scalable eye tracking that has the potential to accelerate research in the fields of psychological science, cognitive neuroscience, user experience design, and human–computer interfaces.

Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurements

We present Captivates, an open-source smartglasses system designed for long-term, in-the-wild psychophysiological monitoring at scale. Captivates integrate many underutilized physiological sensors in a streamlined package, including temple and nose temperature measurement, blink detection, head motion tracking, activity classification, 3D localization, and head pose estimation. Captivates was designed with an emphasis on (1) manufacturing and scalability, so we can easily support large-scale user studies for ourselves and offer the platform as a generalized tool for ambulatory psychophysiology research; (2) robustness and battery life, so long-term studies result in trustworthy data across an individual's entire day in natural environments without supervision or recharge; and (3) aesthetics and comfort, so people can wear them in their normal daily contexts without self-consciousness or changes in behavior.