Eye-based Interaction Research Articles

Eye Facial Gesture System Controller Based on AI ML

Abstract: This paper presents a ground-breaking real-time algorithm for accurately detecting eye blinks in video sequences captured using standard cameras. The algorithm utilizes advanced landmark detectors trained on diverse datasets to ensure robustness against variations in head orientation, illumination, and facial expressions. By employing precise landmark detection techniques, the algorithm estimates the level of eye opening using the eye aspect ratio (EAR) in each frame. An SVM classifier analyses the patterns of EAR values within a short temporal window to identify eye blinks. Comparative evaluations on popular datasets demonstrate the superior performance of the proposed algorithm compared to existing methods. Additionally, the research explores the utilization of eye movements for controlling computer programs, evaluating four distinct approaches in a user-friendly photo viewer. The evaluation process considers various factors such as component sizes, execution time, unintended selections, and gesture repetitions. User experiments reveal that component sizes of 200px provide a convenient and efficient means of application control. The gaze-based method and gestures based on joining points receive positive feedback and exhibit satisfactory performance. This paper contributes significantly to the field of eye-based interaction by introducing an innovative blink detection algorithm and conducting a comprehensive evaluation of eye movement approaches for application control.

Exploring Gaze Movement Gesture Recognition Method for Eye-Based Interaction Using Eyewear with Infrared Distance Sensor Array

With the spread of eyewear devices, people are increasingly using information devices in various everyday situations. In these situations, it is important for eyewear devices to have eye-based interaction functions for simple hands-free input at a low cost. This paper proposes a gaze movement recognition method for simple hands-free interaction that uses eyewear equipped with an infrared distance sensor. The proposed method measures eyelid skin movement using an infrared distance sensor inside the eyewear and applies machine learning to the time-series sensor data to recognize gaze movements (e.g., up, down, left, and right). We implemented a prototype system and conducted evaluations with gaze movements including factors such as movement directions at 45-degree intervals and the movement distance difference in the same direction. The results showed the feasibility of the proposed method. The proposed method recognized 5 to 20 types of gaze movements with an F-value of 0.96 to 1.0. In addition, the proposed method was available with a limited number of sensors, such as two or three, and robust against disturbance in some usage conditions (e.g., body vibration, facial expression change). This paper provides helpful findings for the design of gaze movement recognition methods for simple hands-free interaction using eyewear devices at a low cost.

Comparison of Eye-Based and Controller-Based Selection in Virtual Reality

ABSTRACT Eye tracking, or pointing, in head-mounted displays enables new input modalities for point-select tasks. The goal of this paper is to explore the Fitts’ modeling of the eye-based selection in a virtual reality environment with controller-based input, providing the baseline for two types of eye-based interaction (dwell and physical trigger) in both three-dimensional and two-dimensional environment. In general, the controller-based interaction offered the highest throughput, best accuracy, and preferences of most participants. The eye-trigger interaction performed roughly between the other two. However, the performance difference between the three interaction modes had become smaller when it comes to three-dimensional targets. The performance of eye-movement interactions had been slightly better in terms of accuracy. Generally speaking, eye-based interaction still has a long way to go before becoming one of the mainstream interaction modalities in virtual reality due to the absence of a more stable and precise eye-tracking device with better calibration methods. Still, in some specific virtual reality environments, eye-based interaction has irreplaceable potential.

Eye-Controlled Interfaces for Multimedia Interaction

The EU-funded MAMEM project (Multimedia Authoring and Management using your Eyes and Mind) aims to propose a framework for natural interaction with multimedia information for users who lack fine motor skills. As part of this project, the authors have developed a gaze-based control paradigm. Here, they outline the challenges of eye-controlled interaction with multimedia information and present initial project results. Their objective is to investigate how eye-based interaction techniques can be made precise and fast enough to let disabled people easily interact with multimedia information.

Error Analysis of Line of Sight Estimation using Purkinje Images for Eye-based Human-Computer Interaction: HCI

Error analysis of line of sight estimation using Purkinje images for eye-based Human-Computer Interaction: HCI is conducted. Double Purkinje images which are obtained by two points of light sources are used in the proposed method for eye rotation angle estimation. It aimed at the improvement of the eyeball rotation angle accuracy by simply presuming the value of the curvature radius of the cornea. This technique is a cancellation of the time of the calibration that is the problem of past glance presumption. By presuming the size of the radius of curvature of the cornea. As a result, the eyeball rotation angle presumption accuracy of about 0.98deg was obtained 0.57deg and horizontally in the vertical direction without doing the calibration.

Introduction to the PETMEI special issue

Latest developments in remote and head-mounted eye tracking and automated eye movement analysis point the way toward unobtrusive eye-based human-computer interfaces that will become pervasively usable in everyday life. We call this new paradigm pervasive eye tracking – continuous eye monitoring and analysis 24/7. Pervasive Eye Tracking and Mobile Eye-Based Interaction (PETMEI) is a workshop series that revolves around the theme of pervasive eye-tracking as a trailblazer for pervasive eye-based human-computer interaction and eye-based context-awareness. This special issue is composed from extended versions of the top-scoring papers from the 3rd workshop in the PETMEI series held in 2013.

Estimation of a focused object using a corneal surface image for eye-based interaction

Researchers are considering the use of eye tracking in head-mounted camera systems, such as Google’s Project Glass. Typical methods require detailed calibration in advance, but long periods of use disrupt the calibration record between the eye and the scene camera. In addition, the focused object might not be estimated even if the point-of-regard is estimated using a portable eye-tracker. Therefore, we propose a novel method for estimating the object that a user is focused upon, where an eye camera captures the reflection on the corneal surface. Eye and environment information can be extracted from the corneal surface image simultaneously. We use inverse ray tracing to rectify the reflected image and a scale-invariant feature transform to estimate the object where the point-of-regard is located. Unwarped images can also be generated continuously from corneal surface images. We consider that our proposed method could be applied to a guidance system and we confirmed the feasibility of this application in experiments that estimated the object focused upon and the point-of-regard.

Method for Psychological Status Estimation by Gaze Location Monitoring Using Eye-Based Human- Computer Interaction

Method for psychological status estimation by gaze location monitoring using Eye-Based Human-Computer Interaction: EBHCI is proposed. Through the experiment with English book reading of e-learning content, relation between psychological status and the distance between the correct location of English sentence reading points and the corresponding location derived from EBHCI is clarified. Psych9ological status is estimated from peak alpha frequency derived from eeg signals. It is concluded that psychological status can be estimated with gaze location monitoring.

Eye-based Human Computer Interaction Allowing Phoning, Reading E-Book/E-Comic/E-Learning, Internet Browsing, and TV Information Extraction

Eye-based Human-Computer Interaction: HCI system which allows phoning, reading e-book/e-comic/e-learning, internet browsing, and TV information extraction is proposed for handicap student in E-Learning Application. The conventional eye-based HCI applications are facing problems on accuracy and process speed. We develop new interfaces for improving key-in accuracy and process speed of eye-based key-in for E-Learning application, in particular. We propose eye-based HCI by utilizing camera mounted glasses for gaze estimation. We use the sight for controlling the user interface such as navigation of e-comic/e-book/e-learning contents, phoning, internet browsing, and TV information extraction. We develop interfaces including standard interface navigator with five keys, single line of moving keyboard, and multi line of moving keyboard in order to allow the aforementioned functions without burdening the accuracy. The experimental results show the proposed system does work the aforementioned functions in a real time basis.

Toward Mobile Eye-Based Human-Computer Interaction

Current research on eye-based interfaces mostly focuses on stationary settings. However, advances in mobile eye-tracking equipment and automated eye-movement analysis now allow for investigating eye movements during natural behavior and promise to bring eye-based interaction into people's everyday lives. Recent developments in mobile eye tracking equipment point the way toward unobtrusive human-computer interfaces that will become pervasively usable in everyday life. The potential applications for the further capability to track and analyze eye movements anywhere and anytime calls for new research to develop and understand eye-based interaction in mobile daily life settings.