Eye Gestures Research Articles

Advanced Gaming using Eye Gesture Recognition

Eye gesture recognition represents a advanced technology in the evolution of human-computer interaction (HCI) and for advanced gaming purpose, ‘particularly in the Scenario of today’s industry, equipping delicate hardware such as’ infrared cameras’, ‘depth sensors’, and ‘electrooculography’ (EOG), with the advance software algorithms including Convolutional Neural Networks (CNN), gaze estimation algorithm, and real-Time tracking frameworks like Filters Kalman.’ By including machine learning, deep learning, and computing algorithms, this technology enabled precise interpretation of gestures, establishing interaction between ‘augmented and virtual reality of today’s tech world’, and set the stage for enhanced user experience in intelligent and human Centric industrial system. ‘This type of development marked by primary challenges such as: the accurate eye detection and the creation of a suitable sign language for eye gesture using movements of the pupil. This research focused on the utilization of CNN Algorithm technique to address these challenges, accounting for differentiation in pose, Orientation, Location, and Scale’. The system detects the eye gesture, pre-processing the image extracted from dataset from platform named Kaggle. The subsequent image analysis is performed using ‘python programming’ and OpenCV, utilizing the theories of eye detection and segmentation to enhance the accuracy of system. ‘The proposed methodology also integrates the histogram-based approach to differentiate among various machine learning algorithms and provide the optimal results for eye gesture analysis’

Eye Based Interactive Communication System

Eye-based communication systems, like the Blink-To-Speak language, are essential tools for individuals with motor neuron disorders, enabling them to articulate their needs and emotions. These systems leverage eye movements as a means of interaction, providing a vital communication channel for those with severe speech impairments. By interpreting specific eye gestures, users can effectively convey their thoughts, fostering a sense of independence and improving their quality of life. Many of the existing eye-tracking technologies tend to be complex and costly, limiting their accessibility in low-income regions. Using a mobile phone camera, the system captures real-time video frames to detect and track the user’s eyes, leveraging advanced facial landmark detection. Four primary eye movements—Left, Right, Up, and Blink—form the basis of the Blink-To-Live communication model, allowing users to communicate a range of more than 60 daily commands. By translating sequences of three eye gestures, the system generates readable sentences that are displayed on a screen and can be vocalized through synthesized speech. Unlike other sensor-dependent systems, Blink-To-Live is designed for affordability, flexibility, and ease of use, making it accessible without the need for specialized software or hardware. A prototype has been tested on a range of participants, showing positive results in terms of ease of use, flexibility, and cost-effectiveness

VocalVision: Smart Wheelchair Maintenance with Pressure Sensors and Machine Learning

The Vocal Vision system introduces an innovative approach to enhancing electric wheelchair maintenance and control. It utilizes a network of sensors embedded within the wheelchair's wheels to gather real-time data on tire pressure, temperature, tread wear, and alignment. This data is wirelessly transmitted to a central control unit. Advanced algorithms, incorporating machine learning and predictive analytics, analyze the data to detect irregularities and predict maintenance needs. Users can control direction, speed, and perform complex maneuvers with precision using voice commands and eye gestures. The wheelchair integrates OpenCV for eye gesture recognition and Google Speech Recognition API for voice commands, enabling intuitive control methods. This proposed method introduces a new assistive technology for individuals with disabilities, leveraging cutting-edge technologies.

GazeLink: A multi-language low-cost mobile eye-gesture communication system with large language models for people with amyotrophic lateral sclerosis

Amyotrophic Lateral Sclerosis (ALS) patients who have severe motor and speech impairments mostly rely on their eyes and assistive technology to communicate. However, existing high-tech products are expensive and hard to access, while low-tech products are inefficient and restrictive. To mitigate the limitations, this research proposes GazeLink, a multi-language low-cost mobile application for ALS patients to communicate efficiently with only eye movements. First, the system recognizes user eye gestures like left or up with machine learning and a template-matching algorithm. Then, it converts the eye gestures to words through a keyboard that supports English, Spanish, and Chinese. For efficiency, the system employs Large Language Models (LLMs) to generate a suitable sentence with words typed by the user and the context. Finally, the system provides text-to-speech and social media post services for both verbal and digital eye-gesture communication. Simulations conclude that sentence generation with LLMs can reduce user keystrokes by 81% while maintaining 90% of semantic similarity. Usability studies with 30 participants show that GazeLink can recognize eye gestures with 94.1% accuracy in varying lighting. After rapidly learning the user interface in under 10 attempts, first-time participants typed sentences of various lengths with their eyes at 15.1 words per minute, which is 7.2x faster than the common low-tech solution E-Tran. Experiments demonstrate GazeLinks efficiency, learnability, and accuracy in eye-gesture text entry. The system is extremely affordable (less than $0.1 a month), portable, and easily accessible online. It also supports different users, lighting, smartphones, and languages. Product testing with ALS patients and personalized LLM models will be the next step.

EyeGesener: Eye Gesture Listener for Smart Glasses Interaction Using Acoustic Sensing

The smart glasses market has witnessed significant growth in recent years. The interaction of commercial smart glasses mostly relies on the hand, which is unsuitable for scenarios where both hands are occupied. In this paper, we propose EyeGesener, an eye gesture listener for smart glasses interaction using acoustic sensing. To mitigate the Midas touch problem, we meticulously design eye gestures for interaction as two intentional consecutive saccades in a specific direction without visual dwell. The proposed system is a glass-mounted acoustic sensing system with two pairs of commercial speakers and microphones to sense eye gestures. To capture the subtle movements of the eyelid and surrounding skin induced by eye gestures, we design an Orthogonal Frequency Division Multiplexing (OFDM)-based channel impulse response (CIR) estimation schema that allows two speakers to transmit at the same time and in the same frequency band without collision. We implement eye gesture filtering and adversarial-based eye gesture recognition to identify eye gestures for interaction, filtering out daily eye movements. To address the differences in eye size and facial structure among different users, we employ adversarial training to achieve user-independent eye gesture recognition. We evaluate the performance of our system through experiments on data collected from 16 subjects. The experimental result shows that our system can recognize eight eye gestures with an average F1-score of 0.93, and the false alarm rate of our system is 0.03. We develop an interactive real-time audio-video player based on EyeGesener and then conduct a user study. The result demonstrates the high usability of the proposed system.

S3: Speech, Script and Scene driven Head and Eye Animation

We present S 3 , a novel approach to generating expressive, animator-centric 3D head and eye animation of characters in conversation. Given speech audio, a Directorial script and a cinematographic 3D scene as input, we automatically output the animated 3D rotation of each character's head and eyes. S 3 distills animation and psycho-linguistic insights into a novel modular framework for conversational gaze capturing: audio-driven rhythmic head motion; narrative script-driven emblematic head and eye gestures; and gaze trajectories computed from audio-driven gaze focus/aversion and 3D visual scene salience. Our evaluation is four-fold: we quantitatively validate our algorithm against ground truth data and baseline alternatives; we conduct a perceptual study showing our results to compare favourably to prior art; we present examples of animator control and critique of S 3 output; and present a large number of compelling and varied animations of conversational gaze.

Implementation of Gesture-Based Virtual Keyboard and Mouse

The development of human-computer interaction has seen a major change in the last few years, moving in the direction of more user-friendly and effective interfaces. This tendency has prompted researchers to investigate new input techniques, especially in the field of gesture-based control systems. By utilizing both hand and eye gestures to create a gesture-based virtual mouse and keyboard, this paper introduces a novel method of human-computer interaction. The suggested system offers users a smooth and simple way to interact with computers by combining hand gesture recognition and eye-tracking technology. Users can control cursor movement and keyboard inputs without the use of physical peripherals by using the system to track their hand and eye movements and interpret them as commands. The incorporation of eye-tracking technology improves the system's accuracy and responsiveness, enabling more precise cursor control and interaction with elements displayed on the screen. Hand gesture recognition, on the other hand, gives users an ergonomic and natural way to input data, allowing them to carry out commands with straightforward motions. The architecture of the system combines real-time processing capabilities to guarantee low latency and high responsiveness with sophisticated machine-learning algorithms for gesture recognition. Additionally, users can define their gesture commands according to their needs and preferences because the system is flexible and customizable. The suggested gesture-based virtual mouse and keyboard system has undergone rigorous testing and evaluation, and the results show promising performance and usability in a range of scenarios and applications. This novel method of interacting with computers has the potential to improve user experience, accessibility, and productivity in a variety of computing contexts. Keywords: Human-Computer Interaction, Gesture Recognition, Eye-Tracking

Low-Cost Eye Gesture Communication System for People with Motor Disabilities

Current eye-tracking input systems for people with ALS or other motor impairments are expensive, not robust under sunlight, and require frequent re-calibration and substantial, relatively immobile setups. Eye-gaze transfer (e-tran) boards, a low-tech alternative, are challenging to master and offer slow communication rates. To mitigate the drawbacks of these two status quo approaches, we created Gaze Speak, an eye gesture communication system that runs on a smartphone, and is designed to be low-cost, robust, portable, and easy-to-learn, with a higher communication bandwidth than an e-tran board. Gaze Speak can interpret eye gestures in real time, decode these gestures into predicted utterances, and facilitate communication, with different user interfaces for speakers and interpreters. Our evaluations demonstrate that GazeSpeak is robust, has good user satisfaction, and provides a speed improvement with respect to an e-tran board; we also identify avenues for further improvement to low-cost, low-effort gaze-based communication technologies.

Eye-gesture control of computer systems via artificial intelligence

Background Artificial Intelligence (AI) has the potential to significantly enhance human-computer interactions. This paper introduces a cutting-edge method for computer control using eye-gesture recognition. Methods Our system employs a sophisticated algorithm to accurately interpret eye movements, converting them into actionable commands. This technology not only improves accessibility for individuals with physical impairments, but also offers a more intuitive interaction mode for the general user base. Results We tested our method using a comprehensive dataset and achieved a remarkable accuracy rate of over 99.6283% in translating eye gestures into functional commands. Our system utilizes a variety of tools, including PyCharm, OpenCV, mediapipe, and pyautogui, to achieve these results. Conclusions We discuss potential applications of our technology, such as in the emerging field of gesture-controlled weaponry, which could have significant implications for military and rescue operations. Overall, our work represents a substantial step forward in integrating AI with human-computer interaction, enhancing accessibility, improving user engagement, and unlocking innovative applications for critical industries.

Blink-To-Live eye-based communication system for users with speech impairments

Eye-based communication languages such as Blink-To-Speak play a key role in expressing the needs and emotions of patients with motor neuron disorders. Most invented eye-based tracking systems are complex and not affordable in low-income countries. Blink-To-Live is an eye-tracking system based on a modified Blink-To-Speak language and computer vision for patients with speech impairments. A mobile phone camera tracks the patient’s eyes by sending real-time video frames to computer vision modules for facial landmarks detection, eye identification and tracking. There are four defined key alphabets in the Blink-To-Live eye-based communication language: Left, Right, Up, and Blink. These eye gestures encode more than 60 daily life commands expressed by a sequence of three eye movement states. Once the eye gestures encoded sentences are generated, the translation module will display the phrases in the patient’s native speech on the phone screen, and the synthesized voice can be heard. A prototype of the Blink-To-Live system is evaluated using normal cases with different demographic characteristics. Unlike the other sensor-based eye-tracking systems, Blink-To-Live is simple, flexible, and cost-efficient, with no dependency on specific software or hardware requirements. The software and its source are available from the GitHub repository (https://github.com/ZW01f/Blink-To-Live).

Patient Obligation Perceiver Using Eye Gestures

Abstract: We might have encountered some people who suffer from paralysis, had just undergone surgery, or have strange syndromes (such as Locked-In, Quadriplegia, etc.). They might not be able to speak or move their body. People in these circumstances may find it difficult to communicate, which could have an impact on their health. Consequently, a system that can inform the care taker of their basic needs is necessary. As a result, we can develop a method by which the patient can readily communicate his or her needs through eye gazes. To link these movements with an alphabet that is predefined for each purpose, we can apply the idea of Morse code. This alphabet is then used to find the need in the dataset and create a visual and auditory alert for the nurse/care taker.

A STUDY ON NEURO FUZZY ALGORITHM IMPLEMENTATION ON BCI-UAV CONTROL SYSTEMS

Brain-Computer Interface (BCI) machines are capable of obtaining brain activities by conducting Electroencephalogram tests. Developments on both BCI and Machine Learning allowed various researchers to develop and study various BCI control systems, mainly varying with the algorithm implementation. This research presents a performance analysis of the Adaptive Neuro-Fuzzy Inference System (ANFIS) for BCI control systems for drone maneuverability. Eye gestures were used to generate the EEG data that were captured using the Emotiv INSIGHT Neuroheadset. The obtained data were transferred to the computing hardware using IEEE 802.15 wirless communication protocol (i.e. Bluetooth connectivity); the data are processed using the 5th order Butterworth Band-Pass filtering and heuristic filtering. The filtered dataset is then fed to the ANFIS and a Support Vector Machine (SVM) algorithm, the latter serving as the basis, for training and quadcopter control implementation. Three flight tests were done, hover test, flight command test, and the flight control test, the final test compared the performance of the BCI control system using the ANFIS algorithm to the performance of a traditional handheld remote controller. Results from the initial two tests showed that the ANFIS performed comparably with the SVM, and even about 2% better. The final test showed that the BCI control system had a maximum variance of 4% compared to the handheld controller, where the latter served as the basis. It was found that between Machine Learning algorithms, ANFIS is as capable as the SVM for BCI control systems. Further developments may focus on employing time-series EEG preprocessing techniques.

Sex and Sexuality in Autism Spectrum Disorders: A Scoping Review on a Neglected but Fundamental Issue.

ASD consists of a set of permanent neurodevelopmental conditions, which are studded with social and communication differences, limited interests, and repetitive behaviors. Individuals with ASD have difficulty reading eye gestures and expressions, and may also have stereotyped or repetitive language, excessive adherence to routines, fixed interests, and rigid thinking. However, sexuality in adolescents and young adults with ASD is still a poorly studied and neglected issue. This review aims to evaluate sexual function and behavior in individuals with ASD to foster a greater understanding of this important, although often overlooked, issue. This review was conducted by searching peer-reviewed articles published between 01 June 2000 and 31 May 2022 using the following databases: PubMed, Embase, Cochrane Database, and Web of Science. A comprehensive search was conducted using the terms: "Autism" OR "ASD" AND "Sexuality" OR "Romantic relation" OR "sexual behavior" AND/OR "sexual awareness". After an accurate revision of 214 full manuscripts, 11 articles satisfied the inclusion/exclusion criteria. This review found that, although individuals with ASD may have sexual functioning, their sexuality is characterized by higher prevalence rates of gender dysphoria and inappropriate sexual behavior. Furthermore, sexual awareness is reduced in this patient population, and the prevalence of other variants of sexual orientation (i.e., homosexuality, asexuality, bisexuality, etc.) is higher in adolescents with ASD than in non-autistic peers. Sexual health and education should be included in the care path of patients with ASD in order to improve their quality of life and avoid/reduce inappropriate and risky behaviors.

Head and Eye Egocentric Gesture Recognition for Human-Robot Interaction Using Eyewear Cameras

Non-verbal communication plays a particularly important role in a wide range of scenarios in Human-Robot Interaction (HRI). Accordingly, this work addresses the problem of human gesture recognition. In particular, we focus on head and eye gestures, and adopt an egocentric (first-person) perspective using eyewear cameras. We argue that this egocentric view may offer a number of conceptual and technical benefits over scene- or robot-centric perspectives. A motion-based recognition approach is proposed, which operates at two temporal granularities. Locally, frame-to-frame homographies are estimated with a convolutional neural network (CNN). The output of this CNN is input to a long short-term memory (LSTM) to capture longer-term temporal visual relationships, which are relevant to characterize gestures. Regarding the configuration of the network architecture, one particularly interesting finding is that using the output of an internal layer of the homography CNN increases the recognition rate with respect to using the homography matrix itself. While this work focuses on action recognition, and no robot or user study has been conducted yet, the system has been designed to meet real-time constraints. The encouraging results suggest that the proposed egocentric perspective is viable, and this proof-of-concept work provides novel and useful contributions to the exciting area of HRI.

Development of a Sensing Platform Based on Hands-Free Interfaces for Controlling Electronic Devices

Hands-free interfaces are essential to people with limited mobility for interacting with biomedical or electronic devices. However, there are not enough sensing platforms that quickly tailor the interface to these users with disabilities. Thus, this article proposes to create a sensing platform that could be used by patients with mobility impairments to manipulate electronic devices, thereby their independence will be increased. Hence, a new sensing scheme is developed by using three hands-free signals as inputs: voice commands, head movements, and eye gestures. These signals are obtained by using non-invasive sensors: a microphone for the speech commands, an accelerometer to detect inertial head movements, and an infrared oculography to register eye gestures. These signals are processed and received as the user's commands by an output unit, which provides several communication ports for sending control signals to other devices. The interaction methods are intuitive and could extend boundaries for people with disabilities to manipulate local or remote digital systems. As a study case, two volunteers with severe disabilities used the sensing platform to steer a power wheelchair. Participants performed 15 common skills for wheelchair users and their capacities were evaluated according to a standard test. By using the head control they obtained 93.3 and 86.6%, respectively for volunteers A and B; meanwhile, by using the voice control they obtained 63.3 and 66.6%, respectively. These results show that the end-users achieved high performance by developing most of the skills by using the head movements interface. On the contrary, the users were not able to develop most of the skills by using voice control. These results showed valuable information for tailoring the sensing platform according to the end-user needs.

Characterization of Eye Signals in The Noble Prophetic Hadith

Eye gestures are an important component in the expression of meaning and sign. It is considered one of the most active types of signals and is an expressive language that allows communication to take place more meaningfully. For the ancient Arabs, paying attention to the eyes when communicating and expressing was considered important, which tacitly made them agree on the importance of the eyes in the process of expressing one’s thoughts. This paper aims to reveal the importance of eye gestures that are visible in the facial expressions of the Prophet Muhammad PBUH. Accordingly, the methodology employed is through a qualitative approach using content analysis involving the hadith narrations regarding the eye signals of the Prophet PBUH and the scholars' descriptions of them. The findings of the study showed that eye signals were visible in the facial expressions of the Prophet PBUH which were acceptable to the Arabs and by the psychological state of the recipient. It can be concluded that non-verbal communication was once used by the Prophet PBUH and this study suggests that future conducted studies can link between the practices of the Prophet PBUH and the non-verbal practices of modern society today.

Book review: USER You are logged in as... mustaphataibi My Journals My Profile Log Out JOURNAL CONTENT Search All Browse By Issue By Author By Title Other Journals FONT SIZE INFORMATION For Readers For Authors For Librarians Journal Help OPEN JOURNAL SYSTEMS HOME ABOUT USER HOME ARCHIVES ANNOUNCEMENTS Home > Vol 14, No 1 (2022) Translation & Interpreting Translation &

N/A

ЛИЦО ПОД МАСКОЙ И БЕЗ, ИЛИ ПОЧЕМУ ШУРИН ГЛАЗА ЩУРИТ?

During the pandemic, the mask became a necessary attribute. The article discusses the eye gestures, by which you can guess the smile under the mask. According to student surveys, a smile can be recognized by squinting eyes. However, squinted eyes can be part of the gestures of a smile or a grin, which have different meanings. This uncertainty appears in a Russian proverb that characterizes the typical behavior of relatives towards a son-in-law, in which the gesture of the brother-in-law’s eyes remains incomprehensible: The son-in-law likes to take, the father-in-law loves honor, and the brother-in-law squints his eyes. The mimic gesture of SQUINTING EYES can be considered in its linguistic correlates: shchurit’ glaza, prishchurit’sja, s prishchurom, etc. Lexicographic and corpus analysis of these units makes it possible to identify, in addition to the meaning of physical action, emotional and evaluative meanings not noted in modern dictionaries. Based on this analysis and taking into account folklore materials, an interpretation of the brother-in-law’s behavior in the Russian proverb was proposed. Russian units describing the gesture have different versions of English translations, thus we can suggest their cultural and linguistic specifics, possibly reflecting a gesture characteristic of Russian communicative behavior

Eye Movements and Gestures in Simultaneous and Consecutive Interpreting

Eye Movements and Gestures in Simultaneous and Consecutive Interpreting

Assisting System for Paralyzed Patients Using Eye Gesture

Assisting System for Paralyzed Patients Using Eye Gesture