Eye-head Coordination Research Articles

HeadShift: Head Pointing with Dynamic Control-Display Gain

Head pointing is widely used for hands-free input in head-mounted displays (HMDs). The primary role of head movement in an HMD is to control the viewport based on absolute mapping of head rotation to the 3D environment. Head pointing is conventionally supported by the same 1:1 mapping of input with a cursor fixed in the centre of the view, but this requires exaggerated head movement and limits input granularity. In this work, we propose to adopt dynamic gain to improve ergonomics and precision, and introduce the HeadShift technique. The design of HeadShift is grounded in natural eye-head coordination to manage control of the viewport and the cursor at different speeds. We evaluated HeadShift in a Fitts’ Law experiment and on three different applications in VR, finding the technique to reduce error rate and effort. The findings are significant as they show that gain can be adopted effectively for head pointing while ensuring that the cursor is maintained within a comfortable eye-in-head viewing range.

Properties of Gaze Strategies Based on Eye-Head Coordination in a Ball-Catching Task.

Visual motion information plays an important role in the control of movements in sports. Skilled ball players are thought to acquire accurate visual information by using an effective visual search strategy with eye and head movements. However, differences in catching ability and gaze movements due to sports experience and expertise have not been clarified. Therefore, the purpose of this study was to determine the characteristics of gaze strategies based on eye and head movements during a ball-catching task in athlete and novice groups. Participants were softball and tennis players and college students who were not experienced in ball sports (novice). They performed a one-handed catching task using a tennis ball-shooting machine, which was placed at 9 m in front of the participants, and two conditions were set depending on the height of the ball trajectory (high and low conditions). Their head and eye velocities were detected using a gyroscope and electrooculography (EOG) during the task. Our results showed that the upward head velocity and the downward eye velocity were lower in the softball group than in the tennis and novice groups. When the head was pitched upward, the downward eye velocity was induced from the vestibulo-ocular reflex (VOR) during ball catching. Therefore, it is suggested that skilled ball players have relatively stable head and eye movements, which may lead to an effective gaze strategy. An advantage of the stationary gaze in the softball group could be to acquire visual information about the surroundings other than the ball.

The Impact of Visual Field Size on Eye-Head Coordination During Cognitive Processing

The Impact of Visual Field Size on Eye-Head Coordination During Cognitive Processing

Pose2Gaze: Eye-Body Coordination During Daily Activities for Gaze Prediction From Full-Body Poses.

Human eye gaze plays a significant role in many virtual and augmented reality (VR/AR) applications, such as gaze-contingent rendering, gaze-based interaction, or eye-based activity recognition. However, prior works on gaze analysis and prediction have only explored eye-head coordination and were limited to human-object interactions. We first report a comprehensive analysis of eye-body coordination in various human-object and human-human interaction activities based on four public datasets collected in real-world (MoGaze), VR (ADT), as well as AR (GIMO and EgoBody) environments. We show that in human-object interactions, e.g. pick and place, eye gaze exhibits strong correlations with full-body motion while in human-human interactions, e.g. chat and teach, a person's gaze direction is correlated with the body orientation towards the interaction partner. Informed by these analyses we then present Pose2Gaze- a novel eye-body coordination model that uses a convolutional neural network and a spatio-temporal graph convolutional neural network to extract features from head direction and full-body poses, respectively, and then uses a convolutional neural network to predict eye gaze. We compare our method with state-of-the-art methods that predict eye gaze only from head movements and show that Pose2Gaze outperforms these baselines with an average improvement of 24.0% on MoGaze, 10.1% on ADT, 21.3% on GIMO, and 28.6% on EgoBody in mean angular error, respectively. We also show that our method significantly outperforms prior methods in the sample downstream task of eye-based activity recognition. These results underline the significant information content available in eye-body coordination during daily activities and open up a new direction for gaze prediction.

Real-time continuous EOG-based gaze angle estimation with baseline drift compensation under non-stationary head conditions

Objective:This work addresses an impractical and unnatural constraint that has been generally enforced in state-of-the-art electrooculography (EOG)-based gaze estimation methods, that of maintaining a stationary head pose and position. Specifically, this work proposes an EOG-based gaze angle (GA) estimation method that accommodates natural variations in the user’s head pose and position. The EOG data collected under non-stationary head conditions, which was used in this work to validate the proposed method, is also being made publicly available. Methods:This work generalises a two-eye verging gaze geometrical model to cater for arbitrary head poses and positions, and also models the dynamics of the vestibulo-ocular reflex (VOR), which refers to the eye-head coordination that normally takes place during gaze shifts under unrestrained head conditions. These methods are validated by incorporating them within a published multiple-model framework for GA estimation. Results:When applied to short EOG data segments, a horizontal and vertical GA estimation error of 1.85 ± 0.51° and 2.19 ± 0.62°, respectively, and an eye movement detection and labelling F-score close to 90% were obtained. These results are comparable to those reported previously under stationary head conditions. Conclusion:This work demonstrates that accurate GA estimation and eye movement detection and labelling can be achieved using EOG signals, even when the user’s head is not stationary. Significance:This work eliminates the need for users to maintain a stationary head pose and position, a common constraint in the field, thus introducing an EOG-based GA estimation method that allows users to move their heads naturally.

Information Difference of Transfer Entropies between Head Motion and Eye Movement Indicates a Proxy of Driving.

Visual scanning is achieved via head motion and gaze movement for visual information acquisition and cognitive processing, which plays a critical role in undertaking common sensorimotor tasks such as driving. The coordination of the head and eyes is an important human behavior to make a key contribution to goal-directed visual scanning and sensorimotor driving. In this paper, we basically investigate the two most common patterns in eye-head coordination: "head motion earlier than eye movement" and "eye movement earlier than head motion". We utilize bidirectional transfer entropies between head motion and eye movements to determine the existence of these two eye-head coordination patterns. Furthermore, we propose a unidirectional information difference to assess which pattern predominates in head-eye coordination. Additionally, we have discovered a significant correlation between the normalized unidirectional information difference and driving performance. This result not only indicates the influence of eye-head coordination on driving behavior from a computational perspective but also validates the practical significance of our approach utilizing transfer entropy for quantifying eye-head coordination.

Astronauts eye-head coordination dysfunction over the course of twenty space shuttle flights.

Coordination of motor activity is adapted to Earth's gravity (1 g). However, during space flight the gravity level changes from Earth gravity to hypergravity during launch, and to microgravity (0 g) in orbit. This transition between gravity levels may alter the coordination between eye and head movements in gaze performance. We explored how weightlessness during space flight altered the astronauts' eye-head coordination (EHC) with respect to flight day and target eccentricity. Thirty-four astronauts of 20 Space Shuttle missions had to acquire visual targets with angular offsets of 20°, 30°, and 49°. Measurements of eye, head, and gaze positions collected before and during flight days 1 to 15 indicated changes during target acquisition that varied as a function of flight days and target eccentricity. The in-flight alterations in EHC were presumably the result of a combination of several factors, including a transfer from allocentric to egocentric reference for spatial orientation in absence of a gravitational reference, the generation of slower head movements to attenuate motion sickness, and a decrease in smooth pursuit and vestibulo-ocular reflex performance. These results confirm that humans have several strategies for gaze behavior, between which they switch depending on the environmental conditions.

Effectiveness of Proprioception Training for Patients with Forward Head Posture (FHP) - A Literature Review

Background: Forward head posture (FHP) is a condition marked by the anterior placement of the head, often leading to musculoskeletal issues. Proprioception training, which improves body awareness and movement, has been proposed as a potential intervention for FHP. Methods: A detailed search was conducted using databases such as PubMed, Google Scholar, and Sci-Hub, focusing on studies published since 2013. The selection process involved reviewing titles, abstracts, and full texts. A total of six relevant studies were analysed, including randomized controlled trials, experimental studies, comparative studies, systematic reviews, and case-control studies. Result: A total of 6 articles were selected. The review of six studies underscores the potential of body awareness exercises in alleviating discomfort and enhancing balance for individuals with Forward Head Posture (FHP). Customizing these training methods to individual needs is crucial for maximizing their effectiveness. Conclusion: In conclusion, the effectiveness of proprioception training in addressing Forward Head Posture (FHP) becomes evident through these studies. The exercises, such as head relocation and eye-head coordination, offer a promising approach to alleviating discomfort, improving neck mobility, and enhancing overall body awareness. Embracing proprioception training emerges as a valuable strategy for managing FHP and enhancing overall posture and well-being.

Long-term dance training modifies eye-head coordination in response to passive head impulse.

Long-term dance training is known to improve postural control, especially in challenging postural tasks. However, the effect of dance training on the vestibulo-ocular reflex (VOR) has yet to be properly assessed. This study directly investigated whether VOR parameters are influenced by long-term dance training by testing dancers and controls using the video head impulse test. VOR gains using two of the most common methods (area ratio and instantaneous gains), latency and amplitude of the first saccade, if applicable, were computed. Results revealed a larger VOR gain as measured by area gain and instantaneous gain at 40 ms specifically for left-head impulses, but not right-head impulses. No significant differences in saccade frequency, amplitude, or latency were observed between groups. These differences appear to stem from a modified eye-to-head relationship during high-velocity head impulses in dancers. More specifically, the dancers' eyes lead head movement during passively applied head impulses, which result in higher VOR gain.NEW & NOTEWORTHY This study demonstrates, for the first time, that long-term dance training results in a nonlinear relationship between eye and head velocity within the first milliseconds following passive head impulse. The data also suggest a larger VOR gain in dancers. This finding suggests that dance training may modify eye-head relationship in passive high-frequency head movements. This is of particular interest for vestibular rehabilitation.

Differences in visual stimulation between reading and walking and implications for myopia development.

Visual input plays an important role in the development of myopia (nearsightedness), a visual disorder that blurs vision at far distances. The risk of myopia progression increases with the time spent reading and decreases with outdoor activity for reasons that remain poorly understood. To investigate the stimulus parameters driving this disorder, we compared the visual input to the retina of humans performing two tasks associated with different risks of myopia progression, reading and walking. Human subjects performed the two tasks while wearing glasses with cameras and sensors that recorded visual scenes and visuomotor activity. When compared with walking, reading black text in white background reduced spatiotemporal contrast in central vision and increased it in peripheral vision, leading to a pronounced reduction in the ratio of central/peripheral strength of visual stimulation. It also made the luminance distribution heavily skewed toward negative dark contrast in central vision and positive light contrast in peripheral vision, decreasing the central/peripheral stimulation ratio of ON visual pathways. It also decreased fixation distance, blink rate, pupil size, and head-eye coordination reflexes dominated by ON pathways. Taken together with previous work, these results support the hypothesis that reading drives myopia progression by understimulating ON visual pathways.

EHTask: Recognizing User Tasks From Eye and Head Movements in Immersive Virtual Reality.

Understanding human visual attention in immersive virtual reality (VR) is crucial for many important applications, including gaze prediction, gaze guidance, and gaze-contingent rendering. However, previous works on visual attention analysis typically only explored one specific VR task and paid less attention to the differences between different tasks. Moreover, existing task recognition methods typically focused on 2D viewing conditions and only explored the effectiveness of human eye movements. We first collect eye and head movements of 30 participants performing four tasks, i.e., Free viewing, Visual search, Saliency, and Track, in 15 360-degree VR videos. Using this dataset, we analyze the patterns of human eye and head movements and reveal significant differences across different tasks in terms of fixation duration, saccade amplitude, head rotation velocity, and eye-head coordination. We then propose EHTask - a novel learning-based method that employs eye and head movements to recognize user tasks in VR. We show that our method significantly outperforms the state-of-the-art methods derived from 2D viewing conditions both on our dataset (accuracy of 84.4% versus 62.8%) and on a real-world dataset ( 61.9% versus 44.1%). As such, our work provides meaningful insights into human visual attention under different VR tasks and guides future work on recognizing user tasks in VR.

Task-dependent head-eye coordination during natural fixation

Task-dependent head-eye coordination during natural fixation

Altered sensorimotor processing in irritable bowel syndrome: Evidence for a transdiagnostic pathomechanism in functional somatic disorders.

A recent hypothesis suggests that functional somatic symptoms are due to altered information processing in the brain, with rigid expectations biasing sensorimotor signal processing. First experimental results confirmed such altered processing within the affected symptom modality, e.g., deficient eye-head coordination in patients with functional dizziness. Studies in patients with functional somatic symptoms looking at general, trans-symptomatic processing deficits are sparse. Here, we investigate sensorimotor processing during eye-head gaze shifts in irritable bowel syndrome (IBS) to test whether processing deficits exist across symptom modalities. Study participants were seven patients suffering from IBS and seven age- and gender-matched healthy controls who performed large gaze shifts toward visual targets. Participants performed combined eye-head gaze shifts in the natural condition and with experimentally increased head moment of inertia. Head oscillations as a marker for sensorimotor processing deficits were assessed. Bayes statistics was used to assess evidence for the presence or absence of processing differences between IBS patients and healthy controls. With the head moment of inertia increased, IBS patients displayed more pronounced head oscillations than healthy controls (Bayes Factor 10 = 56.4, corresponding to strong evidence). Patients with IBS show sensorimotor processing deficits, reflected by increased head oscillations during large gaze shifts to visual targets. In particular, patients with IBS have difficulties to adapt to the context of altered head moment of inertia. Our results suggest general transdiagnostic processing deficits in functional somatic disorders.

Complementary feedback control enables effective gaze stabilization in animals

Visually active animals coordinate vision and movement to achieve spectacular tasks. An essential prerequisite to guide agile locomotion is to keep gaze level and stable. Since the eyes, head and body can move independently to control gaze, how does the brain effectively coordinate these distinct motor outputs? Furthermore, since the eyes, head, and body have distinct mechanical constraints (e.g., inertia), how does the nervous system adapt its control to these constraints? To address these questions, we studied gaze control in flying fruit flies (Drosophila) using a paradigm which permitted direct measurement of head and body movements. By combining experiments with mathematical modeling, we show that body movements are sensitive to the speed of visual motion whereas head movements are sensitive to its acceleration. This complementary tuning of the head and body permitted flies to stabilize a broader range of visual motion frequencies. We discovered that flies implement proportional-derivative (PD) control, but unlike classical engineering control systems, relay the proportional and derivative signals in parallel to two distinct motor outputs. This scheme, although derived from flies, recapitulated classic primate vision responses thus suggesting convergent mechanisms across phyla. By applying scaling laws, we quantify that animals as diverse as flies, mice, and humans as well as bio-inspired robots can benefit energetically by having a high ratio between head, body, and eye inertias. Our results provide insights into the mechanical constraints that may have shaped the evolution of active vision and present testable neural control hypotheses for visually guided behavior across phyla.

Influence of predictability on saccade timing in a head impulse VOR suppression task.

Gaze stabilization performance has been shown to be influenced differently when the head is either passively or actively moved in normal healthy participants. However, for a visual fixation suppression task, it remains unknown if the pattern of coordinated head and eye movement is influenced differently by passive or active head movements. We used a suppression head impulse paradigm (SHIMP), where the subject's goal was to maintain gaze stabilized on a visual target that moved with the head during rapid impulsive head movements, to evaluate gaze fixation performance in three conditions: (1) passive-unpredictable where the examiner applied impulsive head yaw rotations with random timing and direction, (2) passive-predictable where the direction of head rotation was announced and then the examiner repeatedly applied impulses in the same direction, and (3) active where the test subject self-generated their head movements. Thirteen young healthy adults performed all three conditions to assess the percentage of early saccades that initiated the gaze shift toward the final visual target position and the latency of first saccades. Early saccades were defined as those occurring within the duration of the head impulse. Results showed that active head impulses generated the greatest percentage of early saccades, followed by predictable and unpredictable. Among the two passive conditions, predictability shortened the first saccade onset latencies. Active condition onset latencies were shorter than in either of the passive conditions, showing a consistent head-leads-eye pattern defining a specific behavioral pattern that could vary across patient groups leading to insights into central neural mechanisms that control eye-head coordination.

Reproduction and Evaluation of Eye-Head Coordination in Automotive System Using Digital Human Technology

Reproduction and Evaluation of Eye-Head Coordination in Automotive System Using Digital Human Technology

Research on the Estimation of Gaze Location for Head-eye Coordination Movement

Abstract Sight is the main source for humans to obtain information from the outside world. Due to the structure of the human eye [1], the range of human sight is limited. For this reason, people need to constantly move their line of sight when observing the surrounding environment and the target, and the movement of the sight is based on the coordinated movement of the head and the eye[2]. Therefore, the key issue for gaze research is how to correctly establish the relationship between head-eye movement and gaze movement. Taking the simulated flight environment as the research background, this paper collects a large number of head-eye images through the designed “three-camera and eight-light source” head-eye data acquisition platform, and proposes a gaze estimation method based on the combination of appearance and features, which effectively combines The relationship of head-eye coordination movement. Then, the ResNet-18 deep residual network structure and the traditional BP neural network structure are used to complete the effective fusion of the head pose and human eye features in the process of capturing the sight target, so as to realize the accurate estimation of the sight drop point, and its average accuracy up to 89.9%.

Detection of scene-irrelevant head movements via eye-head coordination information

BackgroundAccurate motion tracking in head-mounted displays (HMDs) has been widely used in immersive VR interaction technologies. However, tracking the head motion of users at all times is not always desirable. During a session of HMD usage, users may make scene-irrelevant head rotations, such as adjusting the head position to avoid neck pain or responding to distractions from the physical world. To the best of our knowledge, this is the first study that addresses the problem of scene-irrelevant head movements. MethodsWe trained a classifier to detect scene-irrelevant motions using temporal eyehead-coordinated information sequences. To investigate the usefulness of the detection results, we propose a technique to suspend motion tracking in HMDs where scene-irrelevant motions are detected. Results/ConclusionsExperimental results demonstrate that the scene-relevancy of movements can be detected using eye-head coordination information, and that ignoring scene-irrelevant head motions in HMDs improves user continuity without increasing sickness or breaking immersion.

Gaze dynamics with spatiotemporal guided feature descriptor for prediction of driver’s maneuver behavior

At different levels of driving automation, driver’s gaze maintains great indispensable importance on semantic perception of the surround. In this work, we model gaze dynamics and clarify its relationship with driver’s maneuver behaviors from personalized driving style. Firstly, this paper proposes an Occlusion-immune Face Detector (OFD) for facial landmark detection, which can adaptively solve the facial occlusion introduced by the body and glasses frame in the real-world driving scenarios. Meanwhile, an Eye-head Coordination Model is brought up to bridge the error gap in gaze direction through determining eye pose and head pose fused pattern. Then, a vectorized spatiotemporal guidance feature (STGF) descriptor combining gaze accumulation and gaze transition frequency is proposed to construct gaze dynamics within a time window. Finally, we predict driver’s maneuver behavior through STGF descriptor considering different driving styles to clarify the relationship between gaze dynamics and driving maneuver task. Natural driving data are sampled in a dedicated instrumented vehicle testbed, on which 15 drivers with three kind of driving styles participated. Experimental results show that the prediction model achieves the best performance, estimating driver’s behavior an average of 1 s ahead of actual behavior with 83.6% accuracy considering driving style, compared with other approaches.

眼球・頭部協調運動から見た 新たな日常生活動作評価の試み

眼球・頭部協調運動から見た 新たな日常生活動作評価の試み