Blink Research Articles

Biometric Authentication based on Liveness Detection Using Face Landmarks and Deep Learning Model

This paper describes the approach to active liveness detection of the face using facial features and movements. The project aims to create a better method for detecting liveness in real-time on an application programming interface (API) server. The project is built using Python programming with the computer vision libraries OpenCV, dlib and MediaPipe and the deep learning library Tensorflow. There are five modules in active liveness detection progress related to different parts or movements on the face: headshakes, nodding, eye blinks, smiles, and mouths. The functionality of modules runs through face landmarking through dlib and MediaPipe and detection of face features through Tensorflow Convolutional Neural Network (CNN) trained in two different approaches: smile detection and eye-blink detection. The result of implementing face landmarking shows an accurate result through the pre-trained model of MediaPipe and the pre-trained parameter of the dlib 68 landmarking model. And more than 90% classification model accuracy in precision, recall, and f1-score for both trained CNNs in detecting smiles and eyes blinking through the Scikit-Learn classification report. In addition, the prototype API is also implemented using the Python RESTful API library, FastAPI, to test the detection functionality in the prototype Android application. The prototype result is outstanding, as the model excellently requests and retrieves from the API server. The possible research path gives the success of real-time detection on API servers for easy implementation of liveness detection on low-spec client devices.

Formulation and Evaluation of Cetrimide Based In situ Gelling System for Ocular Drug Delivery

Conventional drug delivery system is a traditional type of drug delivery system which manifests poor bioavailability of drug due to tear turn over, tear flow and blinking of eye which causes removal of drug at the ocular surfaces. To overcome from the drawbacks of conventional drug delivery system In- situ gels come into the fashion of ocular drug delivery system for the treatment of ocular diseases which have enhanced bioavailability and better for patient’s compliance. The goal of the present work to express formulation and evaluation of cetrimide in situ gels which have anti-fungal activity used in the treatment of fungal keratitis caused by Fusarium solani. Carbopol 934 is used as viscosity enhancing property. Xanthan gum was used as thickening and stabilizing agent. Sodium alginate was used for gelation ability. The combination of carbopol and xanthan gum showed good gelling capacity and gelation time up to 6hrs period. The developed formulation of cetrimide in situ gels was stable non -irritant which are feasible substitute for conventional drug delivery system such as eye drops by virtue of its stability to increase residence time of drug at the ocular surfaces and provide sustained release of drug with increase bioavailability and better for patient’s compliance.

Anti-Sleep Alarm for Drivers

Abstract: Nowadays it is very difficult to stay active all the time due to busy schedules. Falling asleep at the wheel can lead to serious consequences, accidents, and even death. This situation is much more common than we realize and therefore it is very important to fight this problem. So, to solve this problem, we developed a sleep alarm for drivers. This system alerts the user when he falls asleep at the wheel, thus preventing accidents and saving lives. This system is handy for long-distance travelers and late-night drivers. Designed with a blink sensor, it takes safety measures a step further by designing a system to stop the vehicle when the driver's eyes are closed. The system is developed using Arduino microcontroller. The eye blinking sensor is connected to a microcontroller that continuously monitors the frequency of the driver's eye blinking.If the driver's blink rate falls below a certain threshold, the system warns the driver and sends a signal to the vehicle's controlsystem to stop the vehicle.

Back and front peripersonal space: behavioural and EMG evidence of top–down and bottom–up mechanisms

Previous studies have identified a ‘defensive graded field’ in the peripersonal front space where potential threatening stimuli induce stronger blink responses, mainly modulated by top–down mechanisms, which include various factors, such as proximity to the body, stimulus valence, and social cues. However, very little is known about the mechanisms responsible for representation of the back space and the possible role of bottom–up information. By means of acoustic stimuli, we evaluated individuals’ representation for front and back space in an ambiguous environment that offered some degree of uncertainty in terms of both distance (close vs. far) and front–back egocentric location of sound sources. We aimed to consider verbal responses about localization of sound sources and EMG data on blink reflex. Results suggested that stimulus distance evaluations were better explained by subjective front–back discrimination, rather than real position. Moreover, blink response data were also better explained by subjective front–back discrimination. Taken together, these findings suggest that the mechanisms that dictate blink response magnitude might also affect sound localization (possible bottom–up mechanism), probably interacting with top–down mechanisms that modulate stimuli location and distance. These findings are interpreted within the defensive peripersonal framework, suggesting a close relationship between bottom–up and top–down mechanisms on spatial representation.

Deep learning framework for real-time face recognition using multimodal facial expression with feature optimization and hybrid classification

Smile detection plays key role in recognizing human facial expressions, especially in inter-personal relations. In modern biometrics and security applications, face recognition is widely used. In real-life applications, automated face recognition has always been challenging because people express themselves differently. Real-world scenarios present many challenges, such as low-quality images, temporal variations, and disguises that alter facial characteristics. Face recognition has been accomplished using deep learning frameworks, but there are still several challenges to overcome, such as expression variations and lack of training data. In these methods, margins are used to enforce intra-class compactness and inter-class discrepancy to prevent overfitting. Multimodal facial expression detection is used for real-time face recognition to solve the aforementioned problems. First, we utilizes the Prewitt edge detector for eyes, nose, mouth, and brow detection, and design a deep transfer learning model to extract deep features from face images. Next, feature optimization is done by harpy eagle search optimization (HESO) to selects optimal best among multiple features which reduce data dimensionality problem. Then, we employ a semi-region based convolutional neural network (SR-CNN) to analyzes the facial expressions from multimodalities such as eyes (blinking), nose (shapes), mouth (smiling), and brow (shapes). The analyzed facial expressions and corresponding input images are fed into the benchmark SVM classifier for face recognition. The facial expression based face recognition framework is most suited for real-time scenario like video surveillance. Finally, we validate the performance our framework using benchmark multi-modal Cohn-Kanade and real-time 5671 person images datasets. The simulation results show that the proposed framework can effectively suppress data dimensionality issues and achieve high recognition rate compare to state-of-art frameworks.

Application Specific Reconfigurable Processor for Eyeblink Detection from Dual-Channel EOG Signal

The electrooculogram (EOG) is one of the most significant signals carrying eye movement information, such as blinks and saccades. There are many human–computer interface (HCI) applications based on eye blinks. For example, the detection of eye blinks can be useful for paralyzed people in controlling wheelchairs. Eye blink features from EOG signals can be useful in drowsiness detection. In some applications of electroencephalograms (EEGs), eye blinks are considered noise. The accurate detection of eye blinks can help achieve denoised EEG signals. In this paper, we aimed to design an application-specific reconfigurable binary EOG signal processor to classify blinks and saccades. This work used dual-channel EOG signals containing horizontal and vertical EOG signals. At first, the EOG signals were preprocessed, and then, by extracting only two features, the root mean square (RMS) and standard deviation (STD), blink and saccades were classified. In the classification stage, 97.5% accuracy was obtained using a support vector machine (SVM) at the simulation level. Further, we implemented the system on Xilinx Zynq-7000 FPGAs by hardware/software co-design. The processing was entirely carried out using a hybrid serial–parallel technique for low-power hardware optimization. The overall hardware accuracy for detecting blinks was 95%. The on-chip power consumption for this design was 0.8 watts, whereas the dynamic power was 0.684 watts (86%), and the static power was 0.116 watts (14%).

Evaluation of physiological responses to mental workload in n-back and arithmetic tasks

Working memory tasks, such as n-back and arithmetic tasks, are frequently used in studying mental workload. The present study investigated and compared the sensitivity of several physiological measures at three levels of difficulty of n-back and arithmetic tasks. The results showed significant differences in fixation duration and pupil diameter among three task difficulty levels for both n-back and arithmetic tasks. Pupil diameters increase with increasing mental workload, whereas fixation duration decreases. Blink duration and heart rate (HR) were significantly increased as task difficulty increased in the n-back task, while root mean square of successive differences (RMSSD) and standard deviation of R-R intervals (SDNN) were significantly decreased in the arithmetic task. On the other hand, blink rate and Galvanic Skin Response (GSR) were not sensitive enough to assess the differences in task difficulty for both tasks. All significant physiological measures yielded significant differences between low and high task difficulty except for SDNN. Practitioner summary: This study aimed to assess the sensitivity levels of several physiological measures of mental workload in n-back and arithmetic tasks. It showed that pupil diameter was the most sensitive in both tasks. This study also found that most physiological indices are sensitive to an extreme change in task difficulty levels.

MmDrive: Fine-grained Fatigue Driving Detection Using mmWave Radar

Early detection of fatigue driving is pivotal for the safety of drivers and pedestrians. Traditional approaches mainly employ cameras and wearable sensors to detect fatigue features, which are intrusive to drivers. Recent advances in radio frequency (RF) sensing enable non-intrusive fatigue feature detection from the signal reflected by driver’s body. However, existing RF-based solutions only detect partial or coarse-grained fatigue features, which reduces the detection accuracy. To tackle the above limitations, we propose a mmWave-based fatigue driving detection system, called mmDrive, which can detect multiple fine-grained fatigue features from different body parts. However, achieving accurate detection of various fatigue features during driving encounters practical challenges. Specifically, normal driving activities and driver’s involuntary facial movements inevitably cause interference to fatigue features. Thus, we exploit unique geometric and behavioral characteristics of fatigue features and design effective signal processing methods to remove noises from fatigue-irrelevant activities. Based on the detected fatigue features, we further develop a fatigue determination algorithm to decide the driver’s fatigue state. Extensive experiment results from both simulated and real driving environments show that the average accuracy for detecting nodding and yawning features is about 96%, and the average errors for estimating eye blink, respiration, and heartbeat rates are around 2.21 bpm , 0.54 bpm , and 2.52 bpm , respectively. And the accuracy of the fatigue detection algorithm we proposed reached 97.63%.

A Comprehensive Assessment of Tear-Film-Oriented Diagnosis (TFOD) in a Dacryoadenectomy Dry Eye Model.

Tear film instability is a major cause of dry eye disease. In order to treat patients with short tear film breakup time (TBUT)-type dry eye, the development of tear film stabilizing agents is essential. However, the lack of an appropriate animal model of tear film instability has made drug development difficult. Although rabbit dry eye models have been reported in the past, there are only a few reports that focus on tear film instability. Herein, we assessed the tear film stability of a rabbit dry eye model induced by dacryoadenectomy. A clinical evaluation of the ocular surface, interferometry, and histological assessments of the cornea and conjunctiva were performed. Following the removal of the lacrimal glands, TBUT was shortened significantly, with dimple and random breakup patterns prominently observed. Furthermore, the blink rate in this model increased after dacryoadenectomy, suggesting that this model partially captured the phenotypes of human short TBUT-type dry eye and may be useful as an animal model for investigating potential drug candidates.

Assessment of time to death, brain tissue damage and clinical signs of consciousness in mature Jersey x Holstein cattle receiving a secondary penetrating captive bolt gunshot at the poll or frontal sinus locations

Providing cattle with a good death is an ethical imperative for the dairy industry. When using a penetrating captive bolt gun (PCBG), a secondary (adjunctive) method is currently recommended. Some farms prefer to apply the secondary shot in the poll location whereas others prefer the frontal-sinus location. Unfortunately, little is known about the efficacy of these alternative secondary shot locations. The purpose of this study was to compare clinical signs of consciousness and time to death for 2 different secondary PCBG shot locations in mature Jersey x Holstein cross dairy cattle. Data was collected contemporaneously at 6 dairy operations located in the Midwest U.S. After receiving the first PCBG shot in the frontal-sinus location, cattle requiring euthanasia (n = 46) were randomly assigned to receive a secondary PCBG shot in either the poll (Frontal- Poll) or the frontal-sinus location (Frontal-Frontal) and continuously monitored for clinical signs of consciousness (corneal and palpebral reflex, rhythmic breathing, threat response, etc.) and heartbeat. Signs of consciousness were rarely observed immediately following the first frontal-sinus shot. The median time period when the last heartbeat was heard (7-8 min) did not differ between groups (P = 0.253) nor did gross brain trauma scores (P = 0.247). We conclude the location of the secondary PCBG shot (frontal vs poll) does not seem to impact time to death or gross brain trauma scores. These findings confirm results of previous research involving younger cattle and suggest additional research is needed to better understand the effect of different PCBG locations.

Comparative Study on 2D and 3D User Interface for Eliminating Cognitive Loads in Augmented Reality Repetitive Tasks

The two-dimensional (2D) and three-dimensional (3D) user interfaces have been a prominent topic of augmented reality (AR) research, and their impact on the efficacy and usability of one-time tasks has been extensively examined. As AR is increasingly adopted in industry for repetitive tasks, there is an urgent need for research into the effect of the 2D and 3D user interfaces. In this study, we developed two prototypes of the user interfaces and conducted a comparison study with forty participants to assess their respective influence on cognitive load, perceived usability, and learnability. The results showed that the two user interfaces differed significantly in cognitive load and perceived usability. In particular, the 3D user interfaces exhibited substantially shorter eye blinking durations, shorter eye fixation durations, less dispersed eye gaze areas, lower subjective ratings of cognitive load, and higher overall scores of perceived usability than the 2D user interfaces. However, there was no difference in learnability between the two user interfaces throughout the repetitive tasks.

From Bright to Brightness: Mastering the Management of Bilateral Congenital Cataracts

Congenital cataract is the leading cause of vision loss in newborns and children, and it is caused by an opacity of the eye's lens that occurs at birth or within the first few months of life. Congenital cataracts may develop as unilateral or bilateral. Around 50% of cases of it are considered as idiopathic. A 4-month-old boy was brought by his parents to the Department of Pediatric Ophthalmology and Strabismus in RSUD dr. Zainoel Abidin Banda Aceh with a chief complaint of a white spot on the patient’s right eye. The spot had appeared since birth and became more visible when he was 2 months old. The patient's overall condition was good. Both eyes' visual acuity was blink reflex. The ocular pressure in both eyes was normal. An ophthalmologic examination revealed a fully clouded right eye lens and aphakia in the left eye. The patient had undergone cataract surgery on his left eye on September 6th, 2023. He was diagnosed with Congenital Cataract Oculus Dexter (OD) and Aphakia Oculus Sinister (OS) before surgery. The patient underwent right eye cataract surgery with aspiration irrigation (AI), Primary Posterior Capsulotomy (PPC), and Anterior Vitrectomy (AV) without Intraocular Lens (IOL) implantation within the next two weeks. Following the surgery, the patient's visual function improved to fix and follow the object, the pupils looked round in both eyes, aphakia, and no membrane development. The patient was prescribed glasses for optical rehabilitation.

Childhood blindness from surge of advanced stage retinopathy of prematurity in premature infants during the COVID-19 pandemic: a case series

Background Retinopathy of prematurity (ROP) is a proliferative disease of the ocular retinal vasculature in premature infants, being a leading cause of preventable childhood blindness. The COVID-19 pandemic serves as an additional barrier to access to care and referral, potentially leading to more cases of advanced stage ROP. Although surgery can be done, ROP at this stage is highly associated with poor functional outcome irrespective of surgery. This case series aims to report advanced stage (stage 5) ROP in premature infants during the COVID-19 pandemic. MethodsA retrospective case series of medical records of paediatric patients with ROP was performed at Cipto Mangunkusumo National Referral Hospital between March 1, 2020 and December 31, 2021. This case series included new patients that presented to our clinic from March 2020 to December 2021 with stage 5 ROP. ResultsA total of 19 eyes of 10 patients with stage 5 ROP were included in this study. Mean chronologic age at presentation to our referral centre was 7.0 4.0 months. Upon presentation, all had stage 5 ROP with negative blink reflex to light, and none of the eyes in this cohort underwent surgical intervention. Mean gestational age was 30.3 3.2 weeks and mean birth weight was 1,276.7 366.4 g. ConclusionThere was an observable surge of stage 5 ROP at our referral centre with irreversible blindness. This highlights the need for proper ROP screening in all hospitals in Indonesia to prevent delay in detection and care of advanced stage ROP.

The HPA and SAM axis mediate the impairment of creativity under stress.

With the ever-changing social environment, individual creativity is facing a severe challenge induced by stress. However, little is known regarding the underlying mechanisms by which acute stress affects creative cognitive processing. The current research explored the impacts of the neuroendocrine response on creativity under stress and its underlying cognitive flexibility mechanisms. The enzyme-linked immuno sorbent assay was employed to assess salivary cortisol, which acted as a marker of stress-induced activation of the hypothalamic-pituitary-adrenal (HPA) axis. Eye blink rate (EBR) and pupil diameter were measured as respective indicators of dopamine and noradrenaline released by the activation of the sympathetic-adrenal-medullary (SAM) axis. The Wisconsin card task (WCST) measured cognitive flexibility, while the alternative uses task (AUT) and the remote association task (RAT) measured separately divergent and convergent thinking in creativity. Results showed higher cortisol increments following acute stress induction in the stress group than control group. Ocular results showed that the stress manipulation significantly increased EBR and pupil diameter compared to controls, reflecting increased SAM activity. Further analysis revealed that stress-released cortisol impaired the originality component of the AUT, reducing cognitive flexibility as measured by perseverative errors on the WCST task. Serial mediation analyses showed that both EBR and pupil diameter were also associated with increased perseverative errors leading to poor originality on the AUT. These findings confirm that physiological arousal under stress can impair divergent thinking through the regulation of different neuroendocrine pathways, in which the deterioration of flexible switching plays an important mediating role.

The psychophysiology of Mastermind: Characterizing response times and blinking in a high-stakes television game show.

Television game shows have proven to be a valuable resource for studying human behavior under conditions of high stress and high stakes. However, previous work has focused mostly on choices-ignoring much of the rich visual information that is available on screen. Here, we take a first step to extracting more of this information by investigating the response times and blinking of contestants in the BBC show Mastermind. In Mastermind, contestants answer rapid-fire quiz questions while a camera slowly zooms in on their faces. By labeling contestants' behavior and blinks from 25 episodes, we asked how accuracy, response times, and blinking varied over the course of the game. For accuracy and response times, we tested whether contestants responded more accurately and more slowly after an error-exhibiting the "post-error increase in accuracy" and "post-error slowing" which has been repeatedly observed in the lab. For blinking, we tested whether blink rates varied according to the cognitive demands of the game-decreasing during periods of cognitive load, such as when pondering a response, and increasing at event boundaries in the task, such as the start of a question. In contrast to the lab, evidence for post-error changes in accuracy and response time was weak, with only marginal effects observed. In line with the lab, blinking varied over the course of the game much as we predicted. Overall, our findings demonstrate the potential of extracting dynamic signals from game shows to study the psychophysiology of behavior in the real world.

Vocal Visage: Crafting Lifelike 3D Talking Faces from Static Images and Sound

In the field of computer graphics and animation, the challenge of generating lifelike and expressive talking face animations has historically necessitated extensive 3D data and complex facial motion capture systems. However, this project presents an innovative approach to tackle this challenge, with the primary goal of producing realistic 3D motion coefficients for stylized talking face animations driven by a single reference image synchronized with audio input. Leveraging state-of-the-art deep learning techniques, including generative models, image-to-image translation networks, and audio processing methods, the methodology bridges the gap between static images and dynamic, emotionally rich facial animations. The ultimate aim is to synthesize talking face animations that exhibit seamless lip synchronization and natural eye blinking, thereby achieving an exceptional degree of realism and expressiveness, revolutionizing the realm of computer-generated character interactions.

Blink-related EEG activity measures cognitive load during proactive and reactive driving

Assessing drivers’ cognitive load is crucial for driving safety in challenging situations. This research employed the occurrence of drivers’ natural eye blinks as cues in continuously recorded EEG data to assess the cognitive workload while reactive or proactive driving. Twenty-eight participants performed either a lane-keeping task with varying levels of crosswind (reactive) or curve road (proactive). The blink event-related potentials (bERPs) and spectral perturbations (bERSPs) were analyzed to assess cognitive load variations. The study found that task load during reactive driving did not significantly impact bERPs or bERSPs, possibly due to enduring alertness for vehicle control. The proactive driving revealed significant differences in the occipital N1 component with task load, indicating the necessity to adapt the attentional resources allocation based on road demands. Also, increased steering complexity led to decreased frontal N2, parietal P3, occipital P2 amplitudes, and alpha power, requiring more cognitive resources for processing relevant information. Interestingly, the proactive and reactive driving scenarios demonstrated a significant interaction at the parietal P2 and occipital N1 for three difficulty levels. The study reveals that EEG measures related to natural eye blink behavior provide insights into the effect of cognitive load on different driving tasks, with implications for driver safety.

An artificially-intelligent cornea with tactile sensation enables sensory expansion and interaction

We demonstrate an artificially-intelligent cornea that can assume the functions of the native human cornea such as protection, tactile perception, and light refraction, and possesses sensory expansion and interactive functions. These functions are realized by an artificial corneal reflex arc that is constructed to implement mechanical and light information coding, information processing, and the regulation of transmitted light. Digitally-aligned, long and continuous zinc tin oxide (ZTO) semiconductor fabric patterns were fabricated as the active channels of the artificial synapse, which are non-toxic, heavy-metal-free, low-cost, and ensure superior comprehensive optical properties (transmittance >99.89%, haze <0.36%). Precisely-tuned crystal-phase structures of the ZTO fibers enabled reconfigurable synaptic plasticity, which is applicable to encrypted communication and associative learning. This work suggests new strategies for the tuning of synaptic plasticity and the design of visual neuroprosthetics, and has important implications for the development of neuromorphic electronics and for visual restoration.

Brainstem impairment in obstructive sleep apnoea and the effect of CPAP treatment: an electrophysiological blink reflex study

PurposeThis study aimed to evaluate the functionality of the brainstem structures through the blink reflex (BR) test in patients with obstructive sleep apnoea (OSA) and to assess the effects of continuous positive airway pressure (CPAP) treatment on BR responses.MethodsPatients with moderate-severe OSA and controls underwent BR testing. Patients with OSA who were adherent to CPAP therapy repeated BR testing at 6 months follow-up. CPAP adherence was defined as CPAP use for ≥ 4 hour per night on > 5 nights per week with residual apnoea-hypopnea index less than 5 events per hour.ResultsA total of 22 patients with OSA (86% male, mean age 57.8 ± 10.6 years) and 20 controls (60% male, mean age 55.3 ± 9.3 years) were included. Patients with OSA showed longer right and left R1 latency, as well as delayed right ipsilateral and contralateral R2 latencies compared to controls. Patients with OSA who were compliant with CPAP treatment (n = 16; 88% men, mean age 58.8 ± 9.7 years) showed a significant decrease in latency of the right ipsilateral and contralateral R2 responses at 6 months.ConclusionThis study showed an abnormal pattern of BR responses in patients with OSA, consistent with a significant impairment of brainstem functionality in OSA. CPAP treatment partially improved the BR responses, suggesting the importance of treating OSA.

Deception/Truthful Prediction Based on Facial Feature and Machine Learning Analysis

The Automatic Deception detection refers to the investigative practices used to determine whether person is telling you Truth or lie. Automatic deception detection has been studied extensively as it can be useful in many real-life scenarios in health, justice, and security systems. Many psychological studies have been reported for deception detection. Polygraph testing is a current trending technique to detect deception, but it requires human intervention and training. In recent times, many machine learning based approaches have been applied to detect deceptions. Various modalities like Thermal Imaging, Brain Activity Mapping, Acoustic analysis, eye tracking. Facial Micro expression processing and linguistic analyses are used to detect deception. Machine learning techniques based on facial feature analysis look like a promising path for automatic deception detection. It also works without human intervention. So, it may give better results because it does not affect race or ethnicity. Moreover, one can do covert operation to find deceit using facial video recording. Covert Operation may capture the real personality of deceptive persons. By making combination of various facial features like Facial Emotion, Facial Micro Expressions and Eye blink rate, pupil size, Facial Action Units we can get better accuracy in Deception Detection.