Eye-head Movements Research Articles

Eye Tracking and ROI Detection within a Computer Screen Using a Monocular Camera

The region of interest will change according to the task, even in the same situation. In the study, a method for region of interest detection within a computer screen using a monocular camera is proposed. In contrast to gaze tracking techniques that require particular devices (e.g., an eye tracker and RGB-D device) meanwhile complex calibration, a cheap and more convenient monocular camera is used in this study to solves the eye gaze tracking problem. Firstly, Human face is detected in a real-time video sequence using HoG features. Then, the landmarks around the eyes, which reflect the gaze position, are extracted. Next, the iris centers are detected in the eye region. In order to reduce the gaze error caused by head movement, a three-dimensional head model is proposed to estimate head pose. Finally, the eye region is tracked by calculating the eye vectors and head movement. Experiments were performed to evaluate the face detection, landmarks, iris detection, eye movement estimation, and head pose estimation on databases such as the Hong Kong, BioID, and Boston University head pose databases. Besides, experiments for gaze tracking were performed for a real-time video sequence. Deviation is calculated using Euclidean distance between the real and estimated points. The results show that the method achieves an average error of 1.85∘ with head fixed and 3.58∘ with head movement in the range of −45∘ and 45∘. The requirement is detecting the user’s attention in the screen area. Our method can reach the same level to the other methods, even though the accuracy is not state-of-the-art. Meanwhile, as we all know not only a specific point is concerned but also a region area according to the characteristics of human eye imaging, thus the proposed method can meet the requirements of demand.

Gaze, head and eye movements during somersaults with full twists.

Somersaults with or without twists are the most important elements in sports such as gymnastics or trampolining. Moreover, to perform elements with the highest possible difficulty gymnasts should show good form and execution during the flight phase. In order to ensure perfect body control and a safe landing, gaze behavior has been proven to be crucial for athletes to orientate in the air. As eye movement and head movement are closely coordinated, both must be examined while investigating gaze behavior. The aim of the current study is to analyze athletes' head motion and gaze behavior during somersaults with full twists. 15 skilled trampoline gymnasts performed back straight somersaults with a full twist (back full) on the trampoline. Eye movement and head movement were recorded using a portable eye-tracking device and a motion capture suit. The results indicate that gymnasts use the trampoline bed as a fixation point for orientation and control the back full, whereas the fixation onsets for athletes of a better performance class occur significantly later. A strong coordination between gymnasts' eye movement and head movement could be determined: stabilizing the gaze during the fixation period, the eyes move in combination with the head against the twisted somersault direction to counteract the whole body rotation. Although no significant differences could be found between the performance classes with regard to the maximum axial head rotations and maximum head extensions, there seems to be a trend that less skilled gymnasts need orientation as early as possible resulting in greater head rotation angles but a poorer execution.

Modification of Eye-Head Coordination With High Frequency Random Noise Stimulation.

The vestibulo-ocular reflex (VOR) plays an important role in controlling the gaze at a visual target. Although patients with vestibular hypofunction aim to improve their VOR function, some retain dysfunction for a long time. Previous studies have explored the effects of direct current stimulation on vestibular function; however, the effects of random noise stimulation on eye–head coordination have not previously been tested. Therefore, we aimed to clarify the effects of high frequency noisy vestibular stimulation (HF-nVS) on eye–head coordination related to VOR function. Thirteen healthy young adult participants with no serious disease took part in our study. The current amplitude and density used were 0.4 mA and 0.2 mA/cm2, respectively, with a random noise frequency of 100–640 Hz. The electrodes were located on both mastoid processes. The stimulus duration and fade in/out duration were 600 and 10 s, respectively. Subjects oscillated their head horizontally, gazing at the fixation point, at 1 Hz (0.5 cycles/s) for 30 repetitions. The coordination of eye–head movements was measured by eye-tracking and a motion capture system. Peak-to-peak angles for eye and head movement and deviation of the visual line from the fixation target revealed no significant differences between HF-nVS and sham. The lag time between the eye and head movement with HF-nVS post-stimulation was significantly shorter than that of the sham. We found that HF-nVS can reduce the lag time between eye and head movement and improve coordination, contributing to a clear retinal image. This technique could be applied as a form of VOR training for patients with vestibular hypofunction.

Glut1 Deficiency Syndrome (Glut1DS): State of the art in 2020 and recommendations of the international Glut1DS study group.

Glut1 deficiency syndrome (Glut1DS) is a brain energy failure syndrome caused by impaired glucose transport across brain tissue barriers. Glucose diffusion across tissue barriers is facilitated by a family of proteins including glucose transporter type 1 (Glut1). Patients are treated effectively with ketogenic diet therapies (KDT) that provide a supplemental fuel, namely ketone bodies, for brain energy metabolism. The increasing complexity of Glut1DS, since its original description in 1991, now demands an international consensus statement regarding diagnosis and treatment. International experts (n = 23) developed a consensus statement utilizing their collective professional experience, responses to a standardized questionnaire, and serial discussions of wide‐ranging issues related to Glut1DS. Key clinical features signaling the onset of Glut1DS are eye‐head movement abnormalities, seizures, neurodevelopmental impairment, deceleration of head growth, and movement disorders. Diagnosis is confirmed by the presence of these clinical signs, hypoglycorrhachia documented by lumbar puncture, and genetic analysis showing pathogenic SLC2A1 variants. KDT represent standard choices with Glut1DS‐specific recommendations regarding duration, composition, and management. Ongoing research has identified future interventions to restore Glut1 protein content and function. Clinical manifestations are influenced by patient age, genetic complexity, and novel therapeutic interventions. All clinical phenotypes will benefit from a better understanding of Glut1DS natural history throughout the life cycle and from improved guidelines facilitating early diagnosis and prompt treatment. Often, the presenting seizures are treated initially with antiseizure drugs before the cause of the epilepsy is ascertained and appropriate KDT are initiated. Initial drug treatment fails to treat the underlying metabolic disturbance during early brain development, contributing to the long‐term disease burden. Impaired development of the brain microvasculature is one such complication of delayed Glut1DS treatment in the postnatal period. This international consensus statement should facilitate prompt diagnosis and guide best standard of care for Glut1DS throughout the life cycle.

An Enhanced Eye-Tracking Approach Using Pipeline Computation

Tracking and authentication at a low cost and in a convenient way is an important issue nowadays as opposed to using high cost for specified hardware and complex calibration process. In the paper, the prime concentration is divided into two: (i) eye tracking and (ii) parallelism in execution. First part has been subdivided as: Firstly, human face is taken from video sequence, and this is divided into frames. Eye regions are recognized from image frames. Secondly, iris region and pupils (central point) are being identified. In identifying pupil, the energy intensity and edge strength are taken into account together. Iris and eye corners are considered as tracking points. The famous sinusoidal head model is used for 3-D head shape, and adaptive probability density function (APDF) is proposed for estimating pose in facial features extraction. Thirdly, iris (pupil) tracking is completed by integrating eye vector and head movement information gained from APDF. The second part focuses on reducing execution time (ET) by applying pipeline architecture (PA). So the entire process has been subdivided into three phases. Phase-I is tracking eyes and pupils and preparing an eye vector. Phase-II is completing the calibration process. Phase-III is for matching and feature extraction. PA is integrated to improve performance in terms of ET because three phases are running in parallel. The experiment is done based on CK + image sets. Indeed, by using PA, the ET is reduced by (66.68%) two-thirds. The average accuracy in tracking (91% in left eye pupil (LEP)) and 93.6% in right eye pupil (REP)) is a robust one.

Heterogeneous vestibulocerebellar mossy fiber projections revealed by single axon reconstruction in the mouse.

A significant population of neurons in the vestibular nuclei projects to the cerebellum as mossy fibers (MFs) which are involved in the control and adaptation of posture, eye-head movements, and autonomic function. However, little is known about their axonal projection patterns. We studied the morphology of single axons of medial vestibular nucleus (MVN) neurons as well as those originating from primary afferents, by labeling with biotinylated dextran amine (BDA). MVN axons (n = 35) were classified into three types based on their major predominant termination patterns. The Cbm-type terminated only in the cerebellum (15 axons), whereas others terminated in the cerebellum and contralateral vestibular nuclei (cVN/Cbm-type, 13 axons), or in the cerebellum and ipsilateral vestibular nuclei (iVN/Cbm-type, 7 axons). Cbm- and cVN/Cbm-types mostly projected to the nodulus and uvula without any clear relationship with longitudinal stripes in these lobules. They were often bilateral, and sometimes sent branches to the flocculus and to other vermal lobules. Also, the iVN/Cbm-type projected mainly to the ipsilateral nodulus. Neurons of these types of axons showed different distribution within the MVN. The number of MF terminals of some vestibulocerebellar axons, iVN/Cbm-type axons in particular, and primary afferent axons were much smaller than observed in previously studied MF axons originating from major precerebellar nuclei and the spinal cord. The results demonstrated that a heterogeneous population of MVN neurons provided divergent MF inputs to the cerebellum. The cVN/Cbm- and iVN/Cbm-types indicate that some excitatory neuronal circuits within the vestibular nuclei supply their collaterals to the vestibulocerebellum as MFs.

A brief Review of Deep Learning Based Approaches for Facial Expression and Gesture Recognition Based on Visual Information

Psychological researchers and Many Other Researchers have found that body language of a human can provide substantial information in detecting and interpreting emotions. It could express explicit and implicit information mutually of one’s emotional state and intentions over multi-channel modalities. These imperative channels include eye gaze, head movement, facial expression, body posture and gesture and so on. This learning focuses on detecting emotional states from the body language of the hand and face using computer vision and soft computing techniques. The facial expression and gesture recognition have widely used and challenging task in the present scenario. This paper describes brief review of all approaches which are majorly categorized into two -categories where the first category belongs to Conventional approaches and other based on Deep-learning or Non - Conventional. This paper is basically used as a survey of deep-learning approaches for hand-gesture and Facial Expression Recognitions. We describe all review through the taxonomy of deep-learning approaches proposed architecture details, fusion strategies, datasets, way to treat temporal-dimensions of data, its main features, basic knowledge & general understanding of its challenges

Discharge properties of morphologically identified vestibular neurons recorded during horizontal eye movements in the goldfish.

Computational capability and connectivity are key elements for understanding how central vestibular neurons contribute to gaze-stabilizing eye movements during self-motion. In the well-characterized and segmentally distributed hindbrain oculomotor network of goldfish, we determined afferent and efferent connections along with discharge patterns of descending octaval nucleus (DO) neurons during different eye motions. Based on activity correlated with horizontal eye and head movements, DO neurons were categorized into two complementary groups that either increased discharge during both contraversive (type II) eye (e) and ipsiversive (type I) head (h) movements (eIIhI) or vice versa (eIhII). Matching time courses of slow-phase eye velocity and corresponding firing rates during prolonged visual and head rotation suggested direct causality in generating extraocular motor commands. The axons of the dominant eIIhI subgroup projected either ipsi- or contralaterally and terminated in the abducens nucleus, Area II, and Area I with additional recurrent collaterals of ipsilaterally projecting neurons within the parent nucleus. Distinct feedforward commissural pathways between bilateral DO neurons likely contribute to the generation of eye velocity signals in eIhII cells. The shared contribution of DO and Area II neurons to eye velocity storage likely represents an ancestral condition in goldfish that is clearly at variance with the task separation between mammalian medial vestibular and prepositus hypoglossi neurons. This difference in signal processing between fish and mammals might correlate with a larger repertoire of visuo-vestibular-driven eye movements in the latter species that potentially required a shift in sensitivity and connectivity within the hindbrain-cerebello-oculomotor network. NEW & NOTEWORTHY We describe the structure and function of neurons within the goldfish descending octaval nucleus. Our findings indicate that eye and head velocity signals are processed by vestibular and Area II velocity storage integrator circuitries whereas the velocity-to-position Area I neural integrator generates eye position solely. This ancestral condition differs from that of mammals, in which vestibular neurons generally lack eye position signals that are processed and stored within the nucleus prepositus hypoglossi.

SGaze: A Data-Driven Eye-Head Coordination Model for Realtime Gaze Prediction.

We present a novel, data-driven eye-head coordination model that can be used for realtime gaze prediction for immersive HMD-based applications without any external hardware or eye tracker. Our model (SGaze) is computed by generating a large dataset that corresponds to different users navigating in virtual worlds with different lighting conditions. We perform statistical analysis on the recorded data and observe a linear correlation between gaze positions and head rotation angular velocities. We also find that there exists a latency between eye movements and head movements. SGaze can work as a software-based realtime gaze predictor and we formulate a time related function between head movement and eye movement and use that for realtime gaze position prediction. We demonstrate the benefits of SGaze for gaze-contingent rendering and evaluate the results with a user study.

Egocentric vertigo symptoms diminished by multi-modal neurorehabilitation treatment

Egocentric vertigo symptoms diminished by multi-modal neurorehabilitation treatment

Head-Eye Coordination of Humanoid Robot with Potential Controller

With the advancements in the field of anthropomorphic robots, the interest in generating human-like motions has grown ever so rapidly. A major role in this regard is played by control algorithms that produce appropriate dynamic responses with the robot hardware. In this paper we utilized a potential controller to obtain human-like head-eye movements of a robotic android head. The orientations of the robotic head and eyes were subjected to the same constraints that govern human head and eye orientations to the extent allowed by the mechanical design. On top of the potential controller, visual feedback was used to improve the precision of visual target acquisition. The controller was tested in two scenarios; acquiring a fixed target and tracking a moving target in the field of view of the robot. Head orientations of the robot were compared to those of a human subject for a target tracking experiment. Results indicate that the controller is capable of performing object tracking tasks while exhibiting realistic human-like head orientations.

Referential processing during reading: concurrent recordings of eye movements and head motion.

The present study utilized a new experimental set-up synchronizing eye movements and head motion for investigating referential change occurring in a reading task. We examined the effects of a change in narrative perspective during reading on eye movements and head motion. Forty-four participants read texts on a digital tablet, and both participants' eye movements and head movements were recorded using eye tracker and motion capture. The results showed longer eye fixation duration, longer reading time and decreasing head motions when perspective changed. Recent studies have supported the dynamic engagement hypothesis suggesting that there is a fluctuation in cognitive engagement reflected by postural movements. Our findings on head movements seem to validate this hypothesis of a bodily engagement in reading. The results provided by our study showed that the novel methodology of eye and head movement recordings we used was proven to be informative in studying the reader's embodiment during reading.

Head-eye movement of collegiate baseball batters during fastball hitting.

Successful baseball hitting involves a combination of highly trained perceptual skills and forceful bat swing motions. The purpose of the present study was to quantify the horizontal movement of the head and eyes while baseball batters hit a fastball to clarify a visual strategy for this highly trained interceptive task. Six collegiate baseball players hit a fastball that was launched from a pitching machine. The ball speed was 31.9 m·s-1 for the Slow Ball Task and 40.3 m·s-1 for the Fast Ball Task. Horizontal head movements were analysed using images that were captured by two high-speed video cameras. The Horizontal eye movement was recorded with electrooculography. The angular speed of the horizontal head and eye movements during hitting were divided into four time periods (I-40 = 21–40% of total ball-flight, I-60 = 41–60% of total ball-flight, I-80 = 61–80% of total ball-flight, I-100 = 81–100% of total ball-flight) and analysed using analysis of variance and a Tukey post-hoc multiple-comparison. In the Slow Ball Task, the horizontal angular velocity of the head during I-80 was significantly faster than that during I-40 (p < 0.05). In the Fast Ball Task, the horizontal angular velocity of the head during I-80 was significantly faster than that during I-40 and I-60 (p < 0.05). These results indicated that the tracking motion of the head became faster as the launched ball came close to the batters, but there was no change in the angular tracking motion of the eyes. Therefore, rapid eye movement may not be suitable to accurately estimate the ball’s future location during fastball hitting based on the eye-centered coordinates. Our findings suggest that conventional vision training with a wide range of saccadic or smooth-pursuit eye movements does not reflect the characteristics of tracking strategies during baseball hitting.

An Eye Blink and Head Movement Detection for Computer Vision Syndrome

An Eye Blink and Head Movement Detection for Computer Vision Syndrome

Modeling auditory-visual evoked eye-head gaze shifts in dynamic multisteps.

In dynamic visual or auditory gaze double-steps, a brief target flash or sound burst is presented in midflight of an ongoing eye-head gaze shift. Behavioral experiments in humans and monkeys have indicated that the subsequent eye and head movements to the target are goal-directed, regardless of stimulus timing, first gaze shift characteristics, and initial conditions. This remarkable behavior requires that the gaze-control system 1) has continuous access to accurate signals about eye-in-head position and ongoing eye-head movements, 2) that it accounts for different internal signal delays, and 3) that it is able to update the retinal ( TE) and head-centric ( TH) target coordinates into appropriate eye-centered and head-centered motor commands on millisecond time scales. As predictive, feedforward remapping of targets cannot account for this behavior, we propose that targets are transformed and stored into a stable reference frame as soon as their sensory information becomes available. We present a computational model, in which recruited cells in the midbrain superior colliculus drive eyes and head to the stored target location through a common dynamic oculocentric gaze-velocity command, which is continuously updated from the stable goal and transformed into appropriate oculocentric and craniocentric motor commands. We describe two equivalent, yet conceptually different, implementations that both account for the complex, but accurate, kinematic behaviors and trajectories of eye-head gaze shifts under a variety of challenging multisensory conditions, such as in dynamic visual-auditory multisteps.

The compensation of the vestibulo-ocular reflex during rehabilitation of the patients presenting with vestibular neuritis

The objective of the present study was to elucidate the mechanisms behind the compensation of the vestibular ocular reflex and evaluate the effectiveness of vestibular rehabilitation in the patients presenting with vestibular neuritis (VN) with the application of the video head-impulse test (vHIT) and the dynamic visual acuity test (DVAT). The study included 26 patients with vestibular neuritis whose condition was assessed by scoring based on the dizziness handicap inventory, the dynamic visual acuity test, and the video head-impulse test with the evaluation of saccades and the degree of eye-head movement coordination (gain) before and after the course of vestibular rehabilitation. The study has demonstrated that the course of vestibular rehabilitation of the patients presenting with vestibular neuritis resulted in a significant decrease in the scores of dizziness estimated based on the dizziness handicap inventory and an improvement of dynamic visual acuity in the case of the complete gain recovery as well as in the case of persisting impairment of the gain and the development of sufficient 'covert' saccade. Vestibular rehabilitation was unsuccessful in the patients with persistereduced gain and simultaneous development of 'covert' and 'overt' saccades.

Tracking Gaze and Visual Focus of Attention of People Involved in Social Interaction.

The visual focus of attention (VFOA) has been recognized as a prominent conversational cue. We are interested in estimating and tracking the VFOAs associated with multi-party social interactions. We note that in this type of situations the participants either look at each other or at an object of interest; therefore their eyes are not always visible. Consequently both gaze and VFOA estimation cannot be based on eye detection and tracking. We propose a method that exploits the correlation between eye gaze and head movements. Both VFOA and gaze are modeled as latent variables in a Bayesian switching state-space model (also referred switching Kalman filter). The proposed formulation leads to a tractable learning method and to an efficient online inference procedure that simultaneously tracks gaze and visual focus. The method is tested and benchmarked using two publicly available datasets, Vernissage and LAEO, that contain typical multi-party human-robot and human-human interactions.

Fast gaze-contingent optimal decompositions for multifocal displays

As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict can be created between the vergence and accommodation responses of the viewer. Multifocal HMDs have long been investigated as a potential solution in which multiple image planes span the viewer's accommodation range. Such displays require a scene decomposition algorithm to distribute the depiction of objects across image planes, and previous work has shown that simple decompositions can be achieved in real-time. However, recent optimal decompositions further improve image quality, particularly with complex content. Such decompositions are more computationally involved and likely require better alignment of the image planes with the viewer's eyes, which are potential barriers to practical applications. Our goal is to enable interactive optimal decomposition algorithms capable of driving a vergence- and accommodation-tracked multifocal testbed. Ultimately, such a testbed is necessary to establish the requirements for the practical use of multifocal displays, in terms of computational demand and hardware accuracy. To this end, we present an efficient algorithm for optimal decompositions, incorporating insights from vision science. Our method is amenable to GPU implementations and achieves a three-orders-of-magnitude speedup over previous work. We further show that eye tracking can be used for adequate plane alignment with efficient image-based deformations, adjusting for both eye rotation and head movement relative to the display. We also build the first binocular multifocal testbed with integrated eye tracking and accommodation measurement, paving the way to establish practical eye tracking and rendering requirements for this promising class of display. Finally, we report preliminary results from a pilot user study utilizing our testbed, investigating the accommodation response of users to dynamic stimuli presented under optimal decomposition.

Eye-Head Coordination in 31 Space Shuttle Astronauts during Visual Target Acquisition

Between 1989 and 1995, NASA evaluated how increases in flight duration of up to 17 days affected the health and performance of Space Shuttle astronauts. Thirty-one Space Shuttle pilots participating in 17 space missions were tested at 3 different times before flight and 3 different times after flight, starting within a few hours of return to Earth. The astronauts moved their head and eyes as quickly as possible from the central fixation point to a specified target located 20°, 30°, or 60° off center. Eye movements were measured with electro-oculography (EOG). Head movements were measured with a triaxial rate sensor system mounted on a headband. The mean time to visually acquire the targets immediately after landing was 7–10% (30–34 ms) slower than mean preflight values, but results returned to baseline after 48 hours. This increase in gaze latency was due to a decrease in velocity and amplitude of both the eye saccade and head movement toward the target. Results were similar after all space missions, regardless of length.

11-11 視距離を変化させて指さしさせた時の眼球運動と頭部運動の解析(第11部門 ヒューマンインフォメーション(視覚特性および映像・画像評価))

11-11 視距離を変化させて指さしさせた時の眼球運動と頭部運動の解析(第11部門 ヒューマンインフォメーション(視覚特性および映像・画像評価))