Gaze Movements Research Articles

Instrumented Contact Lens to Detect Gaze Movements Independently of Eye Blinks.

To test on humans an eye tracker based on the use of a contact lens with active infrared sources and to evaluate whether this system can track the eye behind the eyelid. The system is made up of a scleral contact lens embedding two vertical cavity self-emitting lasers (VCSELs) remotely driven by eyewear comprised of electronics and cameras. Tests on humans were carried out on five subjects to assess performance and to determine whether the VCSEL spot could be detected behind the eyelid. The lens was worn and used without difficulty. The device allowed accurate tracking of the eye (0.83° ± 0.59°), and the VCSELs can be detected behind closed eyelids. These results confirm those previously obtained in terms of tracking and demonstrate that the device can be used safely to monitor eye movements even when the eyelids are closed. The VCSELs could be useful for a variety of applications to reduce data missed due to blinks, particularly with regard to interactive systems, fundamental studies on closed eye movements, or finding ways to improve clinical diagnostic precision for disorders of consciousness.

Communication Prototype for Post-Stroke Patients Using Electrooculography (EOG)

Background this prototype is intended for post-stroke patients experiencing disabilities in their daily activities, particularly in communication with others. The challenge they face is difficulty in communication, leading to a diminished quality of life for post-stroke patients. The purpose of innovating this communication aid prototype is to facilitate communication between post-stroke patients and caregivers. The method employed in the post-stroke communication aid prototype utilizes Electrooculography (EOG) signals generated from eye muscle movements during eye gazes, captured by the MaM Sense sensor. The variation in Analog-to-Digital Converter (ADC) values in the MaM Sense sensor is exploited to produce various forms of EOG signals. The resultant command signals from this method are processed by a microcontroller and displayed on a 20 x 4 Character LCD. Testing was conducted on 9 healthy individuals, comprising 5 males and 4 females. To ensure the prototype's functionality, testing was also performed on 1 post-stroke patient. The success rate of MaM Sense sensor readings was 80.5% for the 4 communication modes employed, involving 4 eye gaze movements: right gaze, left gaze, upward gaze, and downward gaze. Thus, the post-stroke communication aid prototype proves effective in assisting communication for post-stroke patients and aiding caregivers in understanding the patients' desires. In the future, a wireless system may be implemented for the acquisition of EOG signals attached to the face to minimize the use of cables.

Scene-aware Foveated Rendering.

We propose a new scene-aware foveated rendering method, which incorporates the scene awareness and characteristics of the human visual system into the mapping-based foveated rendering framework. First, we generate the conservative visual importance map that encodes the visual features of the scene, visual acuity, and gaze motion. Second, we construct the pixel size control map using a convolution kernel method. Third, we utilize the pixel size control map to guide the foveated rendering. At last, a temporal coherent refinement strategy is used to maintain the smooth foveated rendering for the adjacent frames. Compared to the state-of-the-art mapping-based foveated rendering methods using the same compression ratio, our method achieves smaller MSE, higher PSNR, and SSIM in the fovea, periphery, salient regions, and the whole image. We also conducted user studies, and the results proved that the perceptual quality of our method has a high visual similarity with the around truth rendered with the full resolution.

Just-in-time: Gaze guidance in natural behavior.

Natural eye movements have primarily been studied for over-learned activities such as tea-making, sandwich-making, and hand-washing, which have a fixed sequence of associated actions. These studies demonstrate a sequential activation of low-level cognitive schemas facilitating task completion. However, whether these action schemas are activated in the same pattern when a task is novel and a sequence of actions must be planned in the moment is unclear. Here, we recorded gaze and body movements in a naturalistic task to study action-oriented gaze behavior. In a virtual environment, subjects moved objects on a life-size shelf to achieve a given order. To compel cognitive planning, we added complexity to the sorting tasks. Fixations aligned with the action onset showed gaze as tightly coupled with the action sequence, and task complexity moderately affected the proportion of fixations on the task-relevant regions. Our analysis revealed that gaze fixations were allocated to action-relevant targets just in time. Planning behavior predominantly corresponded to a greater visual search for task-relevant objects before the action onset. The results support the idea that natural behavior relies on the frugal use of working memory, and humans refrain from encoding objects in the environment to plan long-term actions. Instead, they prefer just-in-time planning by searching for action-relevant items at the moment, directing their body and hand to it, monitoring the action until it is terminated, and moving on to the following action.

Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring.

Modern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important task. Despite the fact that human operator support systems are becoming more and more widespread, at the moment there is no open-source solution that can monitor this human state based on eye movement monitoring in real time and with high accuracy. Such a method allows the prevention of a large number of potential hazardous situations and accidents in critical industries (nuclear stations, transport systems, and air traffic control). This paper describes the developed method for mental fatigue detection based on human eye movements. We based our research on a developed earlier dataset that included captured eye-tracking data of human operators that implemented different tasks during the day. In the scope of the developed method, we propose a technique for the determination of the most relevant gaze characteristics for mental fatigue state detection. The developed method includes the following machine learning techniques for human state classification: random forest, decision tree, and multilayered perceptron. The experimental results showed that the most relevant characteristics are as follows: average velocity within the fixation area; average curvature of the gaze trajectory; minimum curvature of the gaze trajectory; minimum saccade length; percentage of fixations shorter than 150 ms; and proportion of time spent in fixations shorter than 150 milliseconds. The processing of eye movement data using the proposed method is performed in real time, with the maximum accuracy (0.85) and F1-score (0.80) reached using the random forest method.

Evaluation of Optical Tracking to Distinguish Surgeon Experience During Endoscopic Stone Surgery.

Introduction and Objectives: Optical tracking (OT) has shown potential in assessing surgical skill but has yet to be evaluated for endoscopic urologic surgery. We sought to evaluate the potential for OT to distinguish expert and trainee surgeons during flexible ureteroscopy (fURS) for kidney stone treatment in both simulated and live surgical settings. Methods: We performed OT analysis of six surgeons performing stone localization during fURS in two settings. In the first setting, surgeons were tasked with fiducial localization in three separate kidney phantoms during fURS. In the second setting, surgeons performed stone localization via fURS in five separate patients. Surgeons were categorized as "expert" (n = 3, endourologist, average case volume of >100 fURS per year) or trainee (n = 3, trainee, <100 fURS per year). OT metrics were recorded for both settings using the Microsoft HoloLens 2© as the surgeons viewed the surgical monitor during fURS. Standard OT metrics of experts and trainees were compared, and included: area of eye gaze movement, gaze distance traveled, number of gaze fixation points, percentage of gaze fixation dwell time, and number of saccades. Results: In the simulated setting, the average time for stone localization was greater for trainees compared to experts (318 seconds vs 52 seconds, p < 0.01). Additionally, the mean area of eye gaze movements with area of interest (AOI) was greater for trainees compared to experts (1430 cm2 vs 1060 cm2, p < 0.01). The total gaze distance traveled was also greater for trainees compared to experts (1480 cm vs 730 cm, p < 0.01). In the live surgical setting, average time for stone localization was similar for trainees and experts (74 seconds vs 51 seconds, p = 0.05). The total area of eye gaze movements in AOI was greater for trainees compared to experts (700 cm2 vs 30 cm2, p < 0.01). Additionally, the total gaze distance traveled was greater for trainees compared to experts (14,000 cm vs 680 cm, p < 0.01). This suggests more varied and less point specific concentration on the surgical screen by trainees. There was no difference in percentage of gaze fixation dwell time and number of saccades between experts and trainees in either setting. Conclusions: OT analysis can objectively distinguish surgical experience between experts and trainee surgeons performing fURS in both simulated and live surgical settings. These findings may play a role in future surgical training and skills assessment.

Characteristics of the Reciprocal Movement of Radiographers’ Gaze Based on a Comparison of Entry-Level and Experienced Radiographers

With the development of AI, AI may substitute for physicians in radiological interpretations in the future. The purpose of this paper is to identify factors that could form the foundation for algorithms enabling AI to explore pathological findings. This study focuses on the "angles of gaze trajectory" and "average speeds of gaze movement," and conducts a comparative analysis between entry-level and experienced radiographers. Firstly, based on the results of angles of gaze movement, experienced radiographers exhibited frequent movements around the 180-degree mark compared to entry-level radiographer. Both entrylevel and experienced radiographers commonly displayed frequent movements around 0 degrees and near 350 degrees. Secondly, according to the results of average speeds of gaze movement, entry-level radiographers were found to have faster movement speeds compared to experienced radiographers.

Gaze tracking embedded collaborative robots for automated metrology and reverse engineering

Conventional geometric metrology, or three-dimensional (3D) scanning, and reverse engineering heavily rely on the experience of the operators. With an increasing need for automation, robot arms have been adopted for this task. However, due to the large variety of parts and designs, automated path planning could provide a scanning solution that may overlook the critical area, which could potentially deteriorate the scan results. This article explores the integration of collaborative robotics (cobots) with eye-tracking technology to improve the autonomous 3D scanning process. The primary objective of this study is to enhance the accuracy and efficiency of cobots in 3D scanning, particularly in the capture of functionally critical areas, and to provide a detailed description of regions with complex geometric features. The study develops a framework where the scanning path of the robot-driven scanner is partially guided by eye tracking data, that is, calibrated gaze tracking, to improve the automated 3D scanning process. This framework provides an innovative integration of human gaze movement with automatic robot path planning, providing a new way of human-autonomy teaming. Case studies are presented to present and validate the proposed framework to automatically improve the 3D point cloud collection process, specifically in areas that usually require human manual intervention to capture details.

An action-observation method for studying social perception: a mini-review.

An important aspect of any social interaction involves inferring other people's mental states, intentions, and their likely next actions, by way of facial expression, body posture, eye gaze, and limb movements. An actor's production of actions during social interactions and the observer's perception of these actions are thus closely linked. In this review, we outline an action-observation methodology, which not only allows for separate analyses of production and perception, but also promotes the study of the dynamic interaction between these two sides of every social exchange. We review two lines of research that have benefited from its application. The first line focuses on individuals performing tasks alone and the observation of their actions by other individuals in order to make inferences about their attentional states. The second line of study focused on pairs of individuals performing collaborative tasks in naturalistic settings and the observation of these performances by other individuals. We offer several suggestions for how this methodology can be extended to improve on the limitations of the present studies, as well as some suggestions of how to use this methodology to venture into new territory. Our aim is to inspire future research applications of this methodology in order to advance our understanding of social action production and perception.

Virtual Reality as a support in psychotherapies for children with autism and/or social anxiety: a systematic review

ABSTRACT Purpose The growth of Virtual Reality (VR) applications for therapies has increased and diversified over time. Through VR, patients are encouraged to immerse themselves in virtual environments to interact and socialize. In this article we analyze the therapies that apply VR, as a strategy that addresses health needs, to improve the patient’s quality of life in mental disorders such as anxiety and autism spectrum. Furthermore, we compare the similarities and differences of the therapies, evaluating the efficacy of VR. Finally, we present a review of VR from a technological point of view. Methods The literature search was guided by PRISMA guidelines; study eligibility criteria: year of publication from 2014 to 2022, full-length and open access articles. Articles to improve social skills in children and adolescents with autism, as well as in addressing social anxiety in youth. Exclusion criteria: systematic reviews, meta-analyses, comorbidity, postoperative, and other disorders. Results We considered 14 eligible studies, with a low risk of bias. 78% of VR therapies were implemented immersively and 85% interactively, involving active user participation in virtual environments through voice, gaze, and body movements. VR therapies for children with ASD focus on the development of social skills, with facial expression recognition, emotion recognition, social cognitive development, and verbal and non-verbal communication. VR therapies for social anxiety are effectively treated with meditation and progressive muscle relaxation. Conclusion VR can simplify the lives of those patients who are unable to leave home, due to illness or disability, allowing them to comply with therapy sessions.

SPHEER: a rich dataset of time-resolved gaze and head movements in virtual reality

SPHEER: a rich dataset of time-resolved gaze and head movements in virtual reality

Enhancing social robot's direct gaze expression through vestibulo-ocular movements

Non-verbal communication, especially eye contact, plays a crucial role in human interaction. Integrating eye contact capabilities into robotic behavior presents challenges that require comprehensive interdisciplinary collaboration from the fields of vision science and robotics. This study introduces an innovative methodology to improve the user's perception of the tabletop social robot Haru's direct gaze by explaining how eye and head movements can be managed through simulating visual perception processing. By considering the mechanisms of eye and head movements during visual perception processing, the robot's eye and head gaze behaviors are intended to replicate human gaze movements with Vestibulo-Ocular Movement (VOM) during face-to-face interaction, ultimately enhancing the user's perception of the robot's direct gaze.

Gaze modeling in multi-party dialogues and extraversion expression through gaze aversion control

Gaze control plays a crucial role in the context of conversational robots involving multiple participants. Particularly, gaze aversion is not only important for improving human likeness but is also strongly associated with the expression of a robot's personality and should be accounted for dialogue interactions. While previous studies on gaze models considering multi-party conversations have been conducted, those considering gaze aversion are limited. In this study, we developed a gaze motion generation model, focusing on three features: (1) the gaze target (towards dialogue partners or gaze aversion), (2) the gaze duration, and (3) the eyeball direction during gaze aversion. We extracted the gaze model parameters of two individuals with distinct extraversion from a multimodal three-party dialogue database and created a model that can modulate extraversion expression by setting a gaze aversion ratio parameter. We implemented this model on an android robot, which can reproduce human-like gaze behaviors. Video-based experiments have been conducted where participants watched videos of the robot talking with two humans, replicating the three-party dialogue scenarios used in this study, and evaluated the impressions on the robot's extraversion. The results demonstrated that by controlling gaze aversion parameters, the robot could exhibit various levels of extraversion.

Pilots' Reactions to Different Types of Alerts When Using Head-Up Displays.

INTRODUCTION: The benefits of using head-up displays (HUDs) include reducing head-down time during critical flight phases, enhancing awareness of the external environment, and improving in-flight crew performance. However, the monochromatic nature of HUDs, increased head rotation, and longer gaze movement paths might affect pilots' reactions to different types of alerts.METHODS: Pilot workload and behavior differences were examined between HUD and head-down display (HDD) configurations in three alert scenarios. The study was carried out in an A320 flight simulator and 12 pilots participated.RESULTS: Except for one engine-on-fire scenario, pilot workload when using a HUD was significantly lower than using an HDD. In one engine-on-fire (3.98 s vs. 3.57 s) and one gear-disagree (5.42 s vs. 4.69 s) scenario, pilot response time to alerts using HUD was significantly longer than using an HDD. The angle deviations were significantly smaller when using HUDs in both go-around-under-crosswind (2.67° vs. 3.37°) and one engine-on-fire scenario (1.22° vs. 1.89°).DISCUSSION: The HUD is suitable for a lengthy process of manual flight control inputs, which not only reduces workload but also promotes control accuracy. For tasks that rely on automation, the benefits for workload become less obvious. In addition, head rotation and reorientation of attention adversely affected the response time to non-time-critical warnings and cautions. For instantaneous control with high precision requirements, HUDs did not demonstrate a significant advantage.Zheng Y, Lu Y, Jie Y, Fu S. Pilots' reactions to different types of alerts when using head-up displays. Aerosp Med Hum Perform. 2024; 95(9):688-694.

Basic Study for Grasping the Human Gaze Movement of a Remoting-Ship Operator

Our research focuses on understanding the disparity in risk perception when operating maritime autonomous surface ships (MASS) between onboard and through screens. In our initial step, we conducted experiments using a ship simulator, involving 4 students aspiring to be navigators. They navigated through screens and participated in debriefing sessions. Human gaze movement was utilized to ascertain the ability to comprehend navigational risks solely through observation. Simultaneously, salivary alpha-amylase samples were taken to verify their accurate risk cognition. The key finding indicates that human gaze movement aids in precisely understanding navigational risks, suggesting the feasibility of future ship operations through screens. Our research targets navigators crucial in global logistics, aiming to enhance safety in remote ship operations by demonstrating that risk cognition is achievable through human gaze movement. This breakthrough is vital for the evolving realm of MASS operations.

Physiological and performance metrics during a cardiopulmonary real-time feedback simulation to estimate cognitive load

Multitasking is crucial for First Responders (FRs) in emergency scenarios, enabling them to prioritize and treat victims efficiently. However, research on multitasking and its impact on rescue operations are limited. This study explores the relationship between multitasking, working memory, and the performance of chest compressions during cardiopulmonary resuscitation (CPR). In this experiment, eighteen first-year residents participated in a CPR maneuver using a real-time feedback simulator to learn chest compressions. Different additional secondary tasks were developed and accomplished concurrently with the chest compressions. Heart rate, respiration rate, galvanic skin response, body temperature, eye gaze movements and chest compression performance data were collected. The findings of this study indicated that multitasking impacted chest compression quality for all secondary tasks, showing significance (p-value < 0.05) for the frequency of the chest compressions which worsened in all cases. Additionally, vital signs such as heart rate, respiration rate, and eye gaze speed were also affected during multitasking. Nevertheless, this change on vital signs was different depending on the type of secondary task accomplished. Therefore, as a conclusion, performing multiple tasks during chest compressions affects performance. Understanding cognitive load and its impact on vital signs can aid in training FRs to handle complex scenarios efficiently.

Physiological Data Augmentation for Eye Movement Gaze in Deep Learning

Physiological Data Augmentation for Eye Movement Gaze in Deep Learning

Post-Saccadic Oscillations of the Pupil and Lens Reduce Fixation Stability in Retinitis Pigmentosa and Age-Related Macular Degeneration.

Post-saccadic oscillations (PSOs) reflect movements of gaze that result from motion of the pupil and lens relative to the eyeball rather than eyeball rotations. Here, we analyzed the characteristics of PSOs in subjects with age-related macular degeneration (AMD), retinitis pigmentosa (RP), and normal vision (NV). Our aim was to assess the differences in PSOs between people with vision loss and healthy controls because PSOs affect retinal image stability after each saccade. Participants completed a horizontal saccade task and their gaze was measured using a pupil-based eye tracker. Oscillations occurring in the 80 to 200ms post-saccadic period were described with a damped oscillation model. We compared the amplitude, decay time constant, and frequency of the PSOs for the three different groups. We also examined the correlation between these PSO parameters and the amplitude, peak velocity, and final deceleration of the preceding saccades. Subjects with vision loss (AMD, n = 6, and RP, n = 5) had larger oscillation amplitudes, longer decay constants, and lower frequencies than subjects with NV (n = 7). The oscillation amplitudes increased with increases in saccade deceleration in all three groups. The other PSO parameters, however, did not show consistent correlations with either saccade amplitude or peak velocity. Post-saccadic fixation stability in AMD and RP is reduced due to abnormal PSOs. The differences with respect to NV are not due to differences in saccade kinematics, suggesting that anatomic and neuronal variations affect the suspension of the iris and the lens in the patients' eyes.

Gaze Movements of Helicopter Pilots during Real and Simulated Take-Off and Landing Maneuvers

Most accidents and serious incidents of commercial air transport helicopters occur during standard flight phases, whereby a main cause is pilots’ situational awareness. Enabling pilots to better assess their situational awareness can make an important contribution in reducing the risk of fatal accidents. One approach is to examine a pilot’s gaze behavior with the help of eye tracking. This paper reports the results of a case study with eye tracking measurements during real flight and simulator studies of a standard mission profile. The general gaze behavior is characterized by a dominant, external view, and the airspeed and altitude indicator as the most important flight instruments. A real-world applicability of gaze data obtained in the simulator could be shown.

Participatory design of a virtual reality life review therapy system for palliative care

We present a novel Virtual Reality (VR) life review therapy system. Life review therapy is an intervention used by clinical psychologists, counselors, and social workers for patients under palliative care to systematically examine various life experiences to resolve conflicts and complete life tasks. The goal of the therapy is to narrativize the patient’s life before death in order to come to terms with their mortality. Our VR life review therapy system allows therapists to perform their interventions in customized virtual spaces designed to encourage the patient to reflect and reminisce about their life experiences. Our system is also developed in response to mobility restrictions and COVID-19 pandemic-related quarantine measures that have separated patients under palliative care from family and friends, resulting in emotional and psychological distress. As VR systems related to health are increasingly being used by therapists in their interventions, it is of utmost importance that the health professionals who will be the end users of these systems are involved in the design process to create ethical healthcare technologies. In response, we initiated the participatory design and evaluation of our VR life review therapy system prototype with six clinical psychologists who have practical experience delivering life review therapy to patients in palliative care in the Hong Kong hospital system. In the study, each participant experienced the VR life review therapy system and evaluated its strengths and weaknesses through semi-structured interviews. The qualitative data gathered from these interviews was coded according to nine different categories; avatars, content design, evaluation, haptic sensations, interaction, gesturing and gaze direction, movement and navigation, non-verbal communication, soundtrack. These conceptual categories were chosen based on our analysis of common themes reported by the clinical psychologists’ experience of the system. Feedback from clinical psychologists on each of these categories was incorporated into the redesign and update of the final system. Overall, participants generally evaluated the system as feasible and effective for use with patients under palliative care, while movement and interaction were the features requiring the most improvement.