Head Movement Compensation Research Articles

Head motion-corrected eye gaze tracking with the da Vinci surgical system.

To facilitate the integration of point of gaze (POG) as an input modality for robot-assisted surgery, we introduce a robust head movement compensation gaze tracking system for the da Vinci Surgical System. Previous surgical eye gaze trackers require multiple recalibrations and suffer from accuracy loss when users move from the calibrated position. We investigate whether eye corner detection can reduce gaze estimation error in a robotic surgery context. A polynomial regressor is first used to estimate POG after an 8-point calibration, and then, using another regressor, the POG error from head movement is estimated from the shift in 2D eye corner location. Eye corners are computed by first detecting regions of interest using the You Only Look Once (YOLO) object detector trained on 1600 annotated eye images (open dataset included). Contours are then extracted from the bounding boxes and a derivative-based curvature detector refines the eye corner. Through a user study (n = 24), our corner-contingent head compensation algorithm showed an error reduction in degrees of visual angle of 1.20 (p = 0.037) for the left eye and 1.26 (p = 0.079) for the right compared to the previous gold-standard POG error correction method. In addition, the eye corner pipeline showed a root-mean-squared error of 3.57 (SD = 1.92) pixels in detecting eye corners over 201 annotated frames. We introduce an effective method of using eye corners to correct for eye gaze estimation, enabling the practical acquisition of POG in robotic surgery.

Webcam-based gaze estimation for computer screen interaction.

This paper presents a novel webcam-based approach for gaze estimation on computer screens. Utilizing appearance based gaze estimation models, the system provides a method for mapping the gaze vector from the user's perspective onto the computer screen. Notably, it determines the user's 3D position in front of the screen, using only a 2D webcam without the need for additional markers or equipment. The study presents a comprehensive comparative analysis, assessing the performance of the proposed method against established eye tracking solutions. This includes a direct comparison with the purpose-built Tobii Eye Tracker 5, a high-end hardware solution, and the webcam-based GazeRecorder software. In experiments replicating head movements, especially those imitating yaw rotations, the study brings to light the inherent difficulties associated with tracking such motions using 2D webcams. This research introduces a solution by integrating Structure from Motion (SfM) into the Convolutional Neural Network (CNN) model. The study's accomplishments include showcasing the potential for accurate screen gaze tracking with a simple webcam, presenting a novel approach for physical distance computation, and proposing compensation for head movements, laying the groundwork for advancements in real-world gaze estimation scenarios.

Extended Signal-Space Separation Method for Improved Interference Suppression in MEG.

Objective:Magnetoencephalography (MEG) signals typically reflect a mixture of neuromagnetic fields, subject-related artifacts, external interference and sensor noise. Even inside a magnetically shielded room, external interference can be significantly stronger than brain signals. Methods such as signal-space projection (SSP) and signal-space separation (SSS) have been developed to suppress this residual interference, but their performance might not be sufficient in cases of strong interference or when the sources of interference change over time.Methods:Here we suggest a new method, extended signal-space separation (eSSS), which combines a physical model of the magnetic fields (as in SSS) with a statistical description of the interference (as in SSP). We demonstrate the performance of this method via simulations and experimental MEG data.Results:The eSSS method clearly outperforms SSS and SSP in interference suppression regardless of the extent of a priori information available on the interference sources. We also show that the method does not cause location or amplitude bias in dipole modeling.Conclusion:Our eSSS method provides better data quality than SSP or SSS and can be readily combined with other SSS-based methods, such as spatiotemporal SSS or head movement compensation. Thus, eSSS extends and complements the interference suppression techniques currently available for MEG.Significance:Due to its ability to suppress external interference to the level of sensor noise, eSSS can facilitate single-trial data analysis, exemplified in automated analysis of epileptic data. Such an enhanced suppression is especially important in environments with large interference fields.

Strategies for Gaze Stabilization Critically Depend on Locomotor Speed

Locomotion involves complex combinations of translational and rotational head movements. For gaze stability, this necessitates the interplay of angular and linear vestibulo-ocular reflexes (VOR) as well as the integration of visual feedback about the desired viewing distance. Furthermore, gaze stabilizing systems must be able to cope with vast differences in head motion brought about by changing locomotor speeds and patterns (walking vs. running). The present study investigated horizontal and vertical angular VOR (aVOR) and linear gaze stabilization (lGS) as well as compensation for linear head movements by angular counter rotation of the head during treadmill walking and running at different velocities (0.4 to 2.4 m/s) while fixating either a close (0.5 m) or distant (2.0 m) target. In the horizontal plane, the aVOR predominated throughout all locomotor speeds, whereas the compensation of linear translations was highly variable and generally insufficient. In contrast, in the vertical plane, eye and angular head motion steadily became more in phase with increasing locomotor speed, which served to optimize linear motion compensation. Furthermore, the timing of the vertical aVOR became more automated and independent of visual feedback during faster locomotion. Thus, horizontal and vertical gaze stabilization strategies appear to be considerably different. Whereas horizontal gaze control is likely governed by passive sensorimotor reflexes throughout all locomotor speeds, vertical gaze stabilization switches to an automated feed-forward control at faster locomotion. This switch is presumably driven by efference copies from spinal locomotor commands that were previously shown to govern gaze stabilization in animal models during stereotypic locomotion.

Vection and cybersickness generated by head-and-display motion in the Oculus Rift

Cybersickness is often experienced when viewing virtual environments through head-mounted displays (HMDs). This study examined whether vection (i.e., illusory self-motion) and mismatches between perceived and physical head motions contribute to such adverse experiences. Observers made oscillatory yaw head rotations while viewing stereoscopic optic flow through an Oculus Rift HMD. Vection and cybersickness were measured under 3 conditions of visual compensation for physical head movements: “compensated”, “uncompensated”, and “inversely compensated”. When a nearer aperture was simulated by the HMD, vection was found to be strongest in the “compensated” condition and weakest in the “inversely compensated” condition. However, vection was similar for all 3 conditions during full-field exposures. Cybersickness was most severe for the “inversely compensated” condition, but was not different for the other two conditions. We conclude that mismatches between perceived and physical head-movements can contribute strongly to cybersickness. The relationship between vection and cybersickness is weaker and appears complex.

The Importance of Properly Compensating for Head Movements During MEG Acquisition Across Different Age Groups.

Unlike EEG sensors, which are attached to the head, MEG sensors are located outside the head surface on a fixed external device. Subject head movements during acquisition thus distort the magnetic field distributions measured by the sensors. Previous studies have looked at the effect of head movements, but no study has comprehensively looked at the effect of head movements across age groups, particularly in infants. Using MEG recordings from subjects ranging in age from 3months through adults, here we first quantify the variability in head position as a function of age group. We then combine these measured head movements with brain activity simulations to determine how head movements bias source localization from sensor magnetic fields measured during movement. We find that large amounts of head movement, especially common in infant age groups, can result in large localization errors. We then show that proper application of head movement compensation techniques can restore localization accuracy to pre-movement levels. We also find that proper noise covariance estimation (e.g., during the baseline period) is important to minimize localization bias following head movement compensation. Our findings suggest that head position measurement during acquisition and compensation during analysis is recommended for researchers working with subject populations or age groups that could have substantial head movements. This is especially important in infant MEG studies.

Head movement compensation and multi-modal event detection in eye-tracking data for unconstrained head movements

BackgroundThe complexity of analyzing eye-tracking signals increases as eye-trackers become more mobile. The signals from a mobile eye-tracker are recorded in relation to the head coordinate system and when the head and body move, the recorded eye-tracking signal is influenced by these movements, which render the subsequent event detection difficult. New methodThe purpose of the present paper is to develop a method that performs robust event detection in signals recorded using a mobile eye-tracker. The proposed method performs compensation of head movements recorded using an inertial measurement unit and employs a multi-modal event detection algorithm. The event detection algorithm is based on the head compensated eye-tracking signal combined with information about detected objects extracted from the scene camera of the mobile eye-tracker. ResultsThe method is evaluated when participants are seated 2.6m in front of a big screen, and is therefore only valid for distant targets. The proposed method for head compensation decreases the standard deviation during intervals of fixations from 8° to 3.3° for eye-tracking signals recorded during large head movements. Comparison with existing methodsThe multi-modal event detection algorithm outperforms both an existing algorithm (I-VDT) and the built-in-algorithm of the mobile eye-tracker with an average balanced accuracy, calculated over all types of eye movements, of 0.90, compared to 0.85 and 0.75, respectively for the compared algorithms. ConclusionsThe proposed event detector that combines head movement compensation and information regarding detected objects in the scene video enables for improved classification of events in mobile eye-tracking data.

Head movement compensation in real-time magnetoencephalographic recordings

Neurofeedback- and brain-computer interface (BCI)-based interventions can be implemented using real-time analysis of magnetoencephalographic (MEG) recordings. Head movement during MEG recordings, however, can lead to inaccurate estimates of brain activity, reducing the efficacy of the intervention. Most real-time applications in MEG have utilized analyses that do not correct for head movement. Effective means of correcting for head movement are needed to optimize the use of MEG in such applications. Here we provide preliminary validation of a novel analysis technique, real-time source estimation (rtSE), that measures head movement and generates corrected current source time course estimates in real-time. rtSE was applied while recording a calibrated phantom to determine phantom position localization accuracy and source amplitude estimation accuracy under stationary and moving conditions. Results were compared to off-line analysis methods to assess validity of the rtSE technique. The rtSE method allowed for accurate estimation of current source activity at the source-level in real-time, and accounted for movement of the source due to changes in phantom position. The rtSE technique requires modifications and specialized analysis of the following MEG work flow steps.•Data acquisition•Head position estimation•Source localization•Real-time source estimationThis work explains the technical details and validates each of these steps.

Online and offline tools for head movement compensation in MEG

Magnetoencephalography (MEG) is measured above the head, which makes it sensitive to variations of the head position with respect to the sensors. Head movements blur the topography of the neuronal sources of the MEG signal, increase localization errors, and reduce statistical sensitivity. Here we describe two novel and readily applicable methods that compensate for the detrimental effects of head motion on the statistical sensitivity of MEG experiments. First, we introduce an online procedure that continuously monitors head position. Second, we describe an offline analysis method that takes into account the head position time-series. We quantify the performance of these methods in the context of three different experimental settings, involving somatosensory, visual and auditory stimuli, assessing both individual and group-level statistics. The online head localization procedure allowed for optimal repositioning of the subjects over multiple sessions, resulting in a 28% reduction of the variance in dipole position and an improvement of up to 15% in statistical sensitivity. Offline incorporation of the head position time-series into the general linear model resulted in improvements of group-level statistical sensitivity between 15% and 29%. These tools can substantially reduce the influence of head movement within and between sessions, increasing the sensitivity of many cognitive neuroscience experiments.

Saccadic Compensation for Smooth Eye and Head Movements During Head-Unrestrained Two-Dimensional Tracking

Spatial updating is the ability to keep track of the position of world-fixed objects while we move. In the case of vision, this phenomenon is called spatial constancy and has been studied in head-restraint conditions. During head-restrained smooth pursuit, it has been shown that the saccadic system has access to extraretinal information from the pursuit system to update the objects' position in the surrounding environment. However, during head-unrestrained smooth pursuit, the saccadic system needs to keep track of three different motor commands: the ocular smooth pursuit command, the vestibuloocular reflex (VOR), and the head movement command. The question then arises whether saccades compensate for these movements. To address this question, we briefly presented a target during sinusoidal head-unrestrained smooth pursuit in darkness. Subjects were instructed to look at the flash as soon as they saw it. We observed that subjects were able to orient their gaze to the memorized (and spatially updated) position of the flashed target generally using one to three successive saccades. Similar to the behavior in the head-restrained condition, we found that the longer the gaze saccade latency, the better the compensation for intervening smooth gaze displacements; after about 400 ms, 62% of the smooth gaze displacement had been compensated for. This compensation depended on two independent parameters: the latency of the saccade and the eye contribution to the gaze displacement during this latency period. Separating gaze into eye and head contributions, we show that the larger the eye contribution to the gaze displacement, the better the overall compensation. Finally, we found that the compensation was a function of the head oscillation frequency and we suggest that this relationship is linked to the modulation of VOR gain. We conclude that the general mechanisms of compensation for smooth gaze displacements are similar to those observed in the head-restrained condition.

Video-Based Head Movement Compensation for Novel Haploscopic Eye-Tracking Apparatus

To describe a video-based head-tracking technique to compensate for torsional, horizontal, and vertical in-plane head movements during pupil/iris-tracking video-oculography with a tilting haploscope. Custom software was developed for image acquisition and off-line analysis for a novel haploscopic viewing device. Head movements were constrained to the frontal plane with the use of a bite plate and a forehead rest. Head movements were monitored by tracking black adhesive dots with a white border placed near the subject's inner canthi. Dot and pupil positions were computed using feature detection based on the Hough transform modified for ellipses. With the inter-dot distance and the relative vertical shifts of the dots during head motion, each video frame was rotated and translated to remove the effects of head movement from eye movement data. This technique was verified with a model head and with healthy subjects, who were asked to strain their heads purposefully against the bite plate during the recording. Head-tracker performance during 45 degrees head tilting was also assessed. Validation experiments with the model head indicated a linear relationship, between true and measured positions, with a Pearson's correlation coefficient of R = 1.00. For human subjects, binocular video-oculographic recordings showed essential elimination of head movement artifacts from the recorded eye movements. Tracking black dots placed near the inner canthi is an effective method of compensation for horizontal, vertical, and torsional in-plane head movements during pupil and iris crypt-based video-oculography.

A cascaded scheme for eye tracking and head movement compensation

The authors previously (1995) proposed an efficient method for tracking the eye movements. The proposed algorithm did not address the issue of compensating for head movements. Head movements are normally much slower than eye movements and can be compensated for using another tracking scheme for head position. In this paper, a hybrid method employing the two tracking schemes is developed. To this end, first a measurement model for the compensation of the head movement is formulated and then the overall tracking scheme is implemented by cascading two Kalman filters. The tracking of the iris movement is followed by the compensation of the head movement for each image frame. Experimental results are presented to demonstrate the accuracy aspects and the real-time applicability of the proposed approach.

The interaction of perceived distance with the perceived direction of visual motion during movements of the eyes and the head.

A horizontally moving target was followed by rotation of the eyes alone or by a lateral movement of the head. These movements resulted in the retinal displacement of a vertically moving target from its perceived path, the amplitude of which was determined by the phase and amplitude of the object motion and of the eye or head movements. In two experiments, we tested the prediction from our model of spatial motion (Swanston, Wade, & Day, 1987) that perceived distance interacts with compensation for head movements, but not with compensation for eye movements with respect to a stationary head. In both experiments, when the vertically moving target was seen at a distance different from its physical distance, its perceived path was displaced relative to that seen when there was no error in perceived distance, or when it was pursued by eye movements alone. In a third experiment, simultaneous measurements of eye and head position during lateral head movements showed that errors in fixation were not sufficient to require modification of the retinal paths determined by the geometry of the observation conditions in Experiments 1 and 2.