Real-time Eye Tracking Research Articles

MoTE: Desarrollo de una Herramienta Informática para el Monitoreo del Compromiso Estudiantil

Student engagement in learning is crucial for academic achievement, motivation, and active participation. However, measuring and accounting for student engagement, both in face-to-face and online sessions, remains a challenge for educators. Existing methods, such as self-reports, interviews, and even real-time eye tracking tools, lack a multidimensional engagement approach (i.e., cognitive, affective, and behavioral). In response, this article presents the development of MoTE, a real-time tool for monitoring student engagement. Following a Design-Based Research methodology, we detail the phases of identification of indicators and visualizations to meet the needs of teachers and students, culminating with the proposal of the architecture, and the implementation of an initial prototype. It ends with the evaluation with 146 students in face-to-face and online contexts, providing valuable information on the indicators, dashboards, and functionalities to design an effective tool for monitoring student engagement. This work not only proposes an innovative approach to assessing engagement but also opens avenues for future research and practical applications in education.

A real-time eye movement-based computer interface for people with disabilities

It is costly to develop systems that enable individuals exposed to Amyotrophic Lateral Sclerosis and similar diseases that directly affect the neuromotor ability to communicate with the outside world. In this study, a budget friendly, high-accuracy, software-based, gaze-controlled, real-time virtual keyboard approach that can enable these people to communicate effectively is proposed. The proposed application requires only a computer and a webcam and has a user-friendly interface that meets the basic daily needs of individuals with disabilities. Since the proposed system does not require an extra action such as blinking, it makes it possible to use computers in advanced stage patients who cannot blink their eyes. The application which uses a deep learning-based facial landmark detector, determines the letters the user focuses on the screen and converts thoughts into text. The part of the screen that the user focuses on is determined with a new selection approach inspired by the K-Nearest Neighbors algorithm. This approach, which does not require blinking, offers high speed and accuracy. In the tests, a typing speed of 23.33 characters per minute is achieved with an accuracy rate of 95.12%. It is anticipated that the study will increase computer accessibility for disabled individuals with limited mobility and contribute to the development of real-time eye tracking systems.

Primate eye tracking with carbon-nanotube-paper-composite based capacitive sensors and machine learning algorithms

BackgroundAccurate real-time eye tracking is crucial in oculomotor system research. While the scleral search coil system is the gold standard, its implantation procedure and bulkiness pose challenges. Camera-based systems are affected by ambient lighting and require high computational and electric power. New MethodThis study presents a novel eye tracker using proximity capacitive sensors made of carbon-nanotube-paper-composite (CPC). These sensors detect femtofarad-level capacitance changes caused by primate corneal movement during horizontal and vertical eye rotations. Data processing and machine learning algorithms are evaluated to enhance the accuracy of gaze angle prediction. ResultsThe system performance is benchmarked against the scleral coil during smooth pursuits, saccades tracking, and fixations. The eye tracker demonstrates up to 0.97 correlation with the coil in eye tracking and is capable of estimating gaze angle with a median absolute error as low as 0.30°. ComparisonThe capacitive eye tracker demonstrates good consistency and accuracy in comparison to the gold-standard scleral search coil method. ConclusionsThis lightweight, non-invasive capacitive eye tracker offers potential as an alternative to traditional coil and camera-based systems in oculomotor research and vision science.

Automatic Home Controlling System for Paralyzed Patient using Eye Gestures

Abstract: The system revolutionizes the lives of paralyzed individuals by harnessing the power of eye-tracking technology for seamless control of home appliances. Utilizing OpenCV for robust and real-time eye tracking, the system enables patients to effortlessly interact with their surroundings by focusing on predefined patterns or commands. A user-friendly interface facilitates the establishment of a connection between eye movements and various household devices, including lights, fans, and entertainment systems. This innovative solution empowers individuals with limited mobility to regain independence, simplifying the management of daily routines and living spaces through intuitive gaze-based commands. By providing a novel avenue for communication and control, the system offers paralyzed patients a renewed sense of autonomy, convenience, and improved quality of life.

An Eye Tracking Based Framework for Safety Improvement of Offshore Operations.

Offshore drilling operations consist of complex and high-risk processes. Lack of situational awareness in drilling operations has become an important human factor issue that causes safety accidents. Prolonged work shifts and fatigue are some of the crucial issues that impact performance. Eye tracking technology can be used to distinguish the degree of awareness or alertness of participants that might be related to fatigue or onsite distractions. Oculomotor activity can be used to obtain visual cues that can quantify the drilling operators' situational awareness that might enable us to develop warning alarms to alert the driller. Such systems can help reduce accidents and save non-productive time. In this paper, eye movement char-acteristics were investigated to differentiate the situational awareness between a representa-tive expert and a group of novices using a scenario-based Virtual Reality Drilling Simulator. Significant visual oculomotor activity differences were identified between the expert and the novices that indicate an eye-tracking based system can detect the distraction and alert-ness exhibited by the workers. Results show promise on developing a framework which implements a real-time eye tracking technology in various drilling operations at drilling rigs and Real Time Operation Centers to improve process safety.

Real-time monitoring of driver distraction: State-of-the-art and future insights

Driver distraction and inattention have been found to be major contributors to a large number of serious road crashes. It is evident that distraction reduces to a great extent driver perception levels as well as their decision making capability and the ability of drivers to control the vehicle. An effective way to mitigate the effects of distraction on crash probability, would be through monitoring the mental state of drivers or their driving behaviour and alerting them when they are in a distracted state. Towards that end, in recent years, several inexpensive and effective detection systems have been developed in order to cope with driver inattention. This study endeavours to critically review and assess the state-of-the-art systems and platforms measuring driver distraction or inattention. A thorough literature review was carried out in order to compare and contrast technologies that can be used to detect, monitor or measure driver's distraction or inattention. The systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. The results indicated that in most of the identified studies, driver distraction was measured with respect to its impact to driver behaviour. Real-time eye tracking systems, cardiac sensors on steering wheels, smartphone applications and cameras were found to be the most frequent devices to monitor and detect driver distraction. On the other hand, less frequent and effective approaches included electrodes, hand magnetic rings and glasses.

Cortical Reorganization After Optical Alignment in Strabismic Patients Outside of Critical Period.

To measure visual crowding, an essential bottleneck on object recognition and reliable psychophysical index of cortex organization, in older children and adults with horizontal concomitant strabismus before and after strabismus surgery. Using real-time eye tracking to ensure gaze-contingent display, we examined the peripheral visual crowding effects in older children and adults with horizontal concomitant strabismus but without amblyopia before and after strabismus surgery. Patients were asked to discriminate the orientation of the central tumbling E target letter with flankers arranged along the radial or tangential axis in the nasal or temporal hemifield at different eccentricities (5° or 10°). The critical spacing value, which is the minimum space between the target and the flankers required for correct discrimination, was obtained for comparisons before and after strabismus surgery. Twelve individuals with exotropia (6 males, 21.75 ± 7.29 years, mean ± SD) and 15 individuals with esotropia (6 males, 24.13 ± 5.96 years) participated in this study. We found that strabismic individuals showed significantly larger critical spacing with nasotemporal asymmetry along the radial axis that related to the strabismus pattern, with exotropes exhibiting stronger temporal field crowding and esotropes exhibiting stronger nasal field crowding before surgical alignment. After surgery, the critical spacing was reduced and rebalanced between the nasal and temporal hemifields. Furthermore, the postoperative recovery of stereopsis was associated with the extent of nasotemporal balance of critical spacing. We find that optical realignment (i.e., strabismus surgery) can normalize the enlarged visual crowding effects, a reliable psychophysical index of cortical organization, in the peripheral visual field of older children and adults with strabismus and rebalance the nasotemporal asymmetry of crowding, promoting the recovery of postoperative stereopsis. Our results indicated a potential of experience-dependent cortical organization after axial alignment even for individuals who are out of the critical period of visual development, illuminating the capacity and limitations of optics on sensory plasticity and emphasizing the importance of ocular correction for clinical practice.

Intelligent Vehicle Black Box System Using IoT

Humans are currently involved in numerous transportation-related accidents on the roads. Numerous accidents and fatalities caused by human error are the result of the rising number of vehicles on Indian roads and a lack of enforcement of traffic laws. They also suffer loss of life and valuable property as a result of those accidents. As a result, accidents involving driver inattention to traffic laws and driver fatigue follow. The field of computer vision is actively researching real-time eye detection and tracking. Face alignment can benefit from eye localization and tracking. The duration of eye closure and percentage of eye closure (PEC) can be used to measure driver fatigue. It is based on a hardware system that uses a camera to capture images of the driver in real time and software to monitor the driver's eye in order to prevent accidents. In the event of an accident, GSM and GPS are utilised to track the location of the automobile, and the local hospital and police are alerted. Thanks to IoT technology, this position may always be found in the cloud platform service. The 24/7 Governance is notified to call for emergency assistance by using the push and panic button.

Non-Intrusive Real Time Eye Tracking Using Facial Alignment for Assistive Technologies.

Most affordable eye tracking systems use either intrusive setup such as head-mounted cameras or use fixed cameras with infrared corneal reflections via illuminators. In the case of assistive technologies, using intrusive eye tracking systems can be a burden to wear for extended periods of time and infrared based solutions generally do not work in all environments, especially outside or inside if the sunlight reaches the space. Therefore, we propose an eye-tracking solution using state-of-the-art convolutional neural network face alignment algorithms that is both accurate and lightweight for assistive tasks such as selecting an object for use with assistive robotics arms. This solution uses a simple webcam for gaze and face position and pose estimation. We achieve a much faster computation time than the current state-of-the-art while maintaining comparable accuracy. This paves the way for accurate appearance-based gaze estimation even on mobile devices, giving an average error of around 4.5°on the MPIIGaze dataset [1] and state-of-the-art average errors of 3.9°and 3.3°on the UTMultiview [2] and GazeCapture [3], [4] datasets respectively, while achieving a decrease in computation time of up to 91%.

Eye movements reveal that low confidence precedes deliberation.

Contemporary dual-process models of reasoning maintain that there are two types of thinking -intuitive and deliberative -and that low confidence often leads to deliberation. Previous studies examining the confidence -deliberation relationship have been limited by (1) issues of endogeneity and between-subject comparisons, which we address in this study through debias training and (2) measures of confidence that are taken relatively late in the reasoning process, which we address by measuring confidence via real-time eye-tracking. Self-reported and eye-tracked confidence were both negatively related to deliberative thinking. This finding provides new evidence of the timecourse of the confidence -deliberation relationship and reveals that lowered confidence precedes deliberation.

Performance Analysis of CNN Models for Mobile Device Eye Tracking with Edge Computing

Eye-tracking is a technique used for determining where users are looking and how long they keep their gaze fixed on a particular location. Developments in mobile technology have made mobile applications pervasive; however, eye tracking on mobile devices is still uncommon. This paper proposes a mobile edge computing architecture for eye tracking. We evaluate four lightweight CNN models (LeNet-5, AlexNet, MobileNet, and ShuffleNet) for gaze estimation on mobile devices using a publicly available dataset called GazeCapture. In order to analyse the feasibility of different inference modes such as on-device, edge-based and cloud-based, we conduct an empirical measurement study to quantify inference time, communication time, and resource consumption in these inference modes. Our analysis indicates that while cloud-based inference provides faster predictions, the communication time between the mobile device and the cloud introduces significant latency into the application. This effectively eliminates the ability to perform real-time eye tracking via cloud inference. Furthermore, our findings show that on-device inference performance is limited by energy and memory consumption, making it unsuitable to provide a high-quality user experience. Additionally, we demonstrated that edge-based inference results in a reasonable response time, memory usage, and energy consumption for eye-tracking applications on mobile devices.

An eye tracking based virtual reality system for use inside magnetic resonance imaging systems

Patients undergoing Magnetic Resonance Imaging (MRI) often experience anxiety and sometimes distress prior to and during scanning. Here a full MRI compatible virtual reality (VR) system is described and tested with the aim of creating a radically different experience. Potential benefits could accrue from the strong sense of immersion that can be created with VR, which could create sense experiences designed to avoid the perception of being enclosed and could also provide new modes of diversion and interaction that could make even lengthy MRI examinations much less challenging. Most current VR systems rely on head mounted displays combined with head motion tracking to achieve and maintain a visceral sense of a tangible virtual world, but this technology and approach encourages physical motion, which would be unacceptable and could be physically incompatible for MRI. The proposed VR system uses gaze tracking to control and interact with a virtual world. MRI compatible cameras are used to allow real time eye tracking and robust gaze tracking is achieved through an adaptive calibration strategy in which each successive VR interaction initiated by the subject updates the gaze estimation model. A dedicated VR framework has been developed including a rich virtual world and gaze-controlled game content. To aid in achieving immersive experiences physical sensations, including noise, vibration and proprioception associated with patient table movements, have been made congruent with the presented virtual scene. A live video link allows subject-carer interaction, projecting a supportive presence into the virtual world.

Instruments of Vision

In the age of ubiquitous visual technologies and systems, our perceptive apparatuses are constantly challenged, adapted, and shaped by instruments and machines, rendering the observing body as an active site of knowledge. Your Eye's Motion by Luna is an interactive installation that uses real-time eye-tracking to control a robotic creature named Luna (Figure 1). Materializing eye movements through a wondrous spectacle of light, motion, and color, the observer becomes conscious of her gaze enacted and extended by a robotic counterpart. Building on a diverse set of theories and understandings of vision from the fields of cybernetics, visual studies, embodied mind, and more, the project explores how our perceptual apparatuses and bodies are reconfigured in relation to machines and the environment to afford new ways of seeing. Once we see how observing bodies accommodate feedback from actions to cognition, we can uncover the embodied and affective potential of eye movement as an interface for robotics. The curiosity of Luna invests in this potential, articulating a unity between our embodied percepts and machinic environments to create a "vision machine."

Technical Survey on the Real Time Eye-tracking Pointing Device as a Smart Medical Equipment

본 논문에서 고안된 시선추적 시스템은 루게릭(Lou Gehrig s)이나 각종 근육 관련 질환으로 신체가 부자연스러운 분들을 위해 쉽게 접근할 수 있도록 고안된 안구기반 컴퓨터 입력 장치로, 약 1,700명 정도로 추산되는 국내 루게릭 환자 수와 각종 사고나 질환에 의해 몸을 움직이기 힘든 환자의 수를 합쳐 잠재적 수요만 국내 3만 명에 이르는 사용자를 위한 안구입력 장치이다. 이 안구 입력 장치는 소수의 사용자를 위한 장치이기 때문에 시중에서는 수많은 종류의 상용기기가 제공되고 있어, 이 잠재적 사용자들이 사용하기에는 가격도 비싸고 사용방법도 어려워, 접근성이 많이 떨어지고 있다. 그 이유로는, 각 개인의 경제 사정과 스마트 디바이스에 대한 개별 사용 경험도 조금씩 달라 시중 시선추적 시스템을 사용해보기에는 비용 면이나 사용성 면에서 접근하기가 어려운 경우가 대부분이라 할 수 있다. 이에 따라, 저가의 기기지만 엄선된 부품과 사용하기 편리한 기술을 통해 IT 기기로의 접근성을 개선하여 사용자들에게 쉬운 접근이 가능하도록 하는 시도는 반드시 필요하다. 이에, 본 논문에서는 여러 종류의 시선추적 시스템을 사용했던 사용자들의 자발적 VoC(Voice of Customer)를 통해 기존 시스템의 부족한 점을 개선하고, 사용성 테스트를 통해 이를 만족하는 시스템을 보완/설계함으로써 훨씬 더 많은 사람 및 환자들이 편리하게 사용할 수 있게하고, 기존 PCCR 시스템 대비 계산량을 15배 이상 줄이는 동시에, 시선 오차도 0.5~1도 이내로 보완된 우수한 성능의 시선추적 시스템을 제안한다.

Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays

Eye pupil tracking is important for augmented reality (AR) three-dimensional (3D) head-up displays (HUDs). Accurate and fast eye tracking is still challenging due to multiple driving conditions with eye occlusions, such as wearing sunglasses. In this paper, we propose a system for commercial use that can handle practical driving conditions. Our system classifies human faces into bare faces and sunglasses faces, which are treated differently. For bare faces, our eye tracker regresses the pupil area in a coarse-to-fine manner based on a revised Supervised Descent Method based eye-nose alignment. For sunglasses faces, because the eyes are occluded, our eye tracker uses whole face alignment with a revised Practical Facial Landmark Detector for pupil center tracking. Furthermore, we propose a structural inference-based re-weight network to predict eye position from non-occluded areas, such as the nose and mouth. The proposed re-weight sub-network revises the importance of different feature map positions and predicts the occluded eye positions by non-occluded parts. The proposed eye tracker is robust via a tracker-checker and a small model size. Experiments show that our method achieves high accuracy and speed, approximately 1.5 and 6.5 mm error for bare and sunglasses faces, respectively, at less than 10 ms on a 2.0GHz CPU. The evaluation dataset was captured indoors and outdoors to reflect multiple sunlight conditions. Our proposed method, combined with AR 3D HUDs, shows promising results for commercialization with low crosstalk 3D images.

Investigating Offshore Helicopter Pilots’ Cognitive Load and Physiological Responses during Simulated In-Flight Emergencies

ABSTRACT Objectives: This study was designed to explore pilots’ emergency response performance by collecting real-time physiological and cognitive workload data during pre-defined offshore helicopter scenarios in a flight simulator. Background: Qualification processes require helicopter pilots to complete scheduled simulation training to ensure competency in areas that would otherwise be too dangerous to practice while flying the actual helicopter. Unfortunately, little is known about how effective this simulation training is in regard to cognitive and physiological responses. Methods: Ten qualified pilots completed six flight simulations while real-time eye tracking and physiological data were collected. The pilots were assigned to one of two groups based on their regular flying position (First Officer (FO) or Captain). Results: The findings indicated that there is a significant difference in maximal Index of Cognitive Activity (ICAmax) and maximal heart rate (HRmax) values between the two groups while completing the same in-flight emergency scenarios. Conclusions: Results suggest that FOs found the sessions more cognitively demanding and also indicated they would benefit from learning how Captains conduct their radial scan of instruments. Therefore, it is recommended that visual search patterns are included in initial training, all flight simulator training includes eye tracking, heart rate data are collected, and that future longitudinal research follows new pilots through their training/upgrading process.

REAL TIME EYE TRACKING FOR PASSWORD AUTHENTICATION

The Generality of today’s authentication systems, including password , fingerprint scanners and Personal Identification Number(PIN) entry, are based on one time, static authentication methods which are prone to attacks. Real time eye tracking contributes significantly for Patients who are incapacitated, internet study , gaze based password entry as well as house hold security applications. The use of real time eye tracking and password authentication system has a wider possibility for reliability, but in this case the system must be secure and restrained for Eye Detection and Tracking. For the user’s security and validation (authentication) personal identification numbers are used abroad thereby reducing the chances of being attacked. In static authentication methods , since the password has to be entered physically using PIN’s there are possibilities for tracking the password using shoulder surfing, in order to avoid this spy game of tracking the password , hands off gaze- based PIN entry techniques are used. The smart camera is used to detect the eye position and for Gaze-based PIN Identification thereby doing the data processing. This project is focused on a real time hands off and gaze based PIN entry , detecting the eye position and tracking the eye movement using the smart camera.

REAL TIME EYE-TRACKING FOR PASSWORD AUTHENTICATION

Real-time eye tracking is vital for hand free blink-based password entry. PIN’s (password authentication numbers) are extensively used for security and authentication. PINs are largely used for safety reasons. Password authentication works when a user enters the password using his hands, which might lead to cracking the password through techniques like shoulder surfing or thermal tracking. Authentications with eye blink techniques, does not leave any traces behind as there is no movement or contact with the keyboard during password entry and thus provides a safe environment. Eye tracking helps in locating the image frames sequentially and tracks the center of the eye. This project introduces an application in which we are combining eye blink tracking PIN entry, face detection/recognition and OTP (One Time Password) to avoid shoulder surfing and thermal tracking attacks.

The Use of Eye Tracking in Assessing Visual Attention

In aviation, procedural errors persist despite pilots having the correct knowledge and ability to perform particular flight tasks. These errors can be attributed to diagnostic errors, inaccurate situational assessment and poor hazard awareness. Eye tracking has emerged as a technology that can be used to monitor pilot performance and provide insights into attentional distribution and the underlying decisional processes. The study used a systematic literature review based on the Preferred Reporting Items for systematic reviews and Meta-Analyses (PRIMA) Statement. Relevant publications were searched from 25 databases. To search the databases, the study used the keywords “eye tracking”, “eye tracking measurements”, “eye tracking aviation safety” and “eye tracking pilot flight performance”, as well as “eye tracking and pilot training” and “eye tracking and pilot assessment”. From these databases, 48 publications were retrieved for analysis. Out of the 48, 33 were abstracts and 15 were full-texts. The 15 full-texts proceeded to systematic literature analysis. These studies were further divided to studies reporting the use of eye tracking as a training tool (7) and studies reporting the use of eye tracking as an assessment tool (8). The literature review demonstrates that eye tracking is a valid and effective tool that can provide quantitative data for flight performance and safety assessment. Continued growth in aircraft cockpit automation means that eye tracking remains an urgent and promising development in aviation research, especially in real-time eye tracking to predict and prevent pilot errors and in the integration of eye tracking technology in intelligent cockpit design.

An integrated highly synchronous, high resolution, real time eye tracking system for dynamic flight movement

Electronic surveillance systems are being used rapidly today, ranging from a simple video camera to a complex biometric surveillance system for facial patterns and intelligent computer vision based surveillance systems, which are applied in many fields such as home monitoring, security surveillance of important places and mission critical tasks like air traffic control surveillance. Such systems normally involve a computer system and a human surveillance operator, who looks at the dynamic display to perform his surveillance tasks. Exploitation of shared information between these physical heterogeneous data capture systems with human operated functions is one emerging aspect in electronic surveillance that has yet to be addressed deeply. Hence, an innovative interaction interface for such knowledge extraction and representation is required. Such an interface should establish a data activity register frame which captures information depicting various surveillance activities at a specified spatial and time reference.This paper presents a real time eye tracking system, which integrates two sets of activity data in a highly dynamic changing and synchronous manner in real-time with respect to both spatial and time frames, through the “Dynamic Data Alignment and Timestamp Synchronisation Model”. This model matches the timestamps of the two data streams, aligns them to the same spatial reference frame before fusing them together into a data activity register frame. The Air Traffic Control (ATC) domain is used to illustrate this model, where experiments are conducted under simulated radar traffic situations with participants and their radar input data. Test results revealed that this model is able to synchronise the timestamp of the eye and dynamic display data, align both of these data spatially, while taking into account dynamic changes in space and time on a simulated radar display. This system can also distinguish and show variations in the monitoring behaviour of participants. As such, new knowledge can be extracted and represented through this innovative interface, which can then be applied to other applications in the field of electronic surveillance to unearth monitoring behaviour of the human surveillance operator.