Eye-hand Movements Research Articles

Changes in Timing and kinematics of goal directed eye-hand movements in early-stage Parkinson’s disease

ObjectiveMany daily activities involve intrinsic or extrinsic goal-directed eye and hand movements. An extensive visuomotor coordination network including nigro-striatal pathways is required for efficient timing and positioning of eyes and hands. The aim of this study was to investigate how Parkinson’s disease (PD) affects eye-hand coordination in tasks with different cognitive complexity.MethodsWe used a touch screen, an eye-tracking device and a motion capturing system to quantify changes in eye-hand coordination in early-stage PD patients (H&Y < 2.5) and age-matched controls. Timing and kinematics of eye and hand were quantified in four eye-hand coordination tasks (pro-tapping, dual planning, anti-tapping and spatial memory task).ResultsIn the pro-tapping task, saccade initiation towards extrinsic goals was not impaired. However, in the dual planning and anti-tapping task initiation of saccades towards intrinsic goals was faster in PD patients. Hand movements were differently affected: initiation of the hand movement was only delayed in the pro-tapping and dual planning task. Overall, hand movements in PD patients were slower executed compared to controls.InterpretationWhereas initiation of saccades in an extrinsic goal-directed task (pro-tapping task) is not affected, early stage PD patients have difficulty in suppressing reflexive saccades towards extrinsic goals in tasks where the endpoint is an intrinsic goal (e.g. dual planning and anti-tapping task). This is specific for eye movements, as hand movements have delayed responses in the pro-tapping and dual planning task. This suggests that reported impairment of the dorsolateral prefrontal cortex in early-stage PD patients affects only inhibition of eye movements. We conclude that timing and kinematics of eye and hand movements in visuomotor tasks are affected in PD patients. This result may have clinical significance by providing a behavioral marker for the early diagnosis of PD.

Very Slow Search and Reach: Failure to Maximize Expected Gain in an Eye-Hand Coordination Task

We examined an eye-hand coordination task where optimal visual search and hand movement strategies were inter-related. Observers were asked to find and touch a target among five distractors on a touch screen. Their reward for touching the target was reduced by an amount proportional to how long they took to locate and reach to it. Coordinating the eye and the hand appropriately would markedly reduce the search-reach time. Using statistical decision theory we derived the sequence of interrelated eye and hand movements that would maximize expected gain and we predicted how hand movements should change as the eye gathered further information about target location. We recorded human observers' eye movements and hand movements and compared them with the optimal strategy that would have maximized expected gain. We found that most observers failed to adopt the optimal search-reach strategy. We analyze and describe the strategies they did adopt.

P1-20: The Relation of Eye and Hand Movement during Multimodal Recall Memory

Eye and hand movement tracking has been proven to be a successful tool and is widely used to figure out characteristics of human cognition in language or visual processing (Just & Carpenter, 1976 Cognitive Psychology8441–480). Eye movement has proven to be a successful measure to figure out characteristics of human language and visual processing (Rayner, 1998 Psychological Bulletin124(3) 372–422). Recently, mouse tracking was used for social-cognition-like categorization of sex-atypical faces and studying spoken-language processes (Magnuson, 2005 PNAS102(28) 9995–9996; Spivey et al., 2005 PNAS102 10393–10398). Here, we present a framework that uses both eye gaze and hand movement simultaneously for analyzing the relation of them during memory retrieval. We tracked eye and mouse movements when the subject was watching a drama and playing a multimodal memory game (MMG), a cognitive task designed to investigate the recall memory mechanisms in watching video dramas (Zhang, 2009 AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers 144–149). Experimental results show that eye tracking and mouse tracking provide complementary information about underlying cognitive processes. Also, we found some interesting patterns in eye-hand movement during multimodal memory recall.

Effect of Aging on Coordinated Eye and Hand Movements With Two-Segment Sequence

This study examined how aging compromises coordinative eye-hand movements with multiple segments. Older adults and young controls performed two-segment movements with the eyes only or with the eyes and hand together. The results showed minimal age-related changes on the initiation and execution of primary saccade during the first segment. However, the older adults showed a scaling problem of saccade velocity when hand movements were included. They were also slow in stabilizing gaze fixation to the first target. Regarding hand movements, the older adults pronouncedly increased the deceleration phase compared with the controls while fixating their gazes to the target. They also increased the intersegment interval for both eye and hand movements. Taken together, aging differentially affects various components of movements, which contributes to the slowness of overall performance.

Does delay impair localisation in blindsight?

The unconscious sensorimotor skills which survive compromise of the geniculostriate visual pathway have been linked with activity of the dorsal stream of extrastriate occipitoparietal cortex. These sensorimotor circuits are thought to operate in real time. Therefore, an introduction of a delay between visual stimulus presentation and the patient's subsequent motor response should severely compromise sensorimotor tasks such as localisation (moving hand or eye to the location of a previously presented visual target). We tested this hypothesis in patient DB, a well-studied case of blindsight whose localisation abilities were first documented in the 1970s. Using eye tracking and hand movement recording technologies, as well as stimuli that control for light scatter, we verified the original observations of DB's manual and saccadic localisation. Remarkably, the introduction of a 4s delay did not compromise his ability to localise with either eye or hand. A control experiment reveals that this skill does not depend on an opportunity to make a decision at the time of stimulus presentation, circumventing the delay using memory. These data suggest that DB's manual and saccadic localisation skills do not depend on the circuits of the dorsal stream, or that delay, contrary to theory, does not severely compromise dorsal sensorimotor skills.

Improving the sensitivity of EEG-fMRI studies of epileptic activity by modelling eye blinks, swallowing and other video-EEG detected physiological confounds

To improve the sensitivity and specificity of simultaneous electroencephalography and functional magnetic resonance imaging (EEG-fMRI) it is prudent to devise modelling strategies explaining the residual variance. The purpose of this study is to investigate the potential value of including additional regressors for physiological activities, derived from video-EEG, in the modelling of haemodynamic patterns linked to interictal epileptiform discharges (IEDs) using simultaneously recorded video-EEG-fMRI. Ten patients with IED (focal epilepsy: 6, idiopathic generalized epilepsy (IGE):4) were studied. BOLD-sensitive fMRI images were acquired on a 3T MRI scanner. 64-channel EEG was recorded using MR-compatible system. A custom made, dual-video-camera system synchronised with EEG was used to record video simultaneously. IEDs and physiological activities were identified and labelled on video-EEG using Brain Analyzer2. fMRI time-series data were pre-processed and analysed using SPM5 software. Two general linear models (GLM) were created; GLM1: IEDs were convolved with the canonical haemodynamic response function and its derivatives. Realignment parameters and pulse regressors were included in the design matrix as confounds, GLM2: GLM1 and additional regressors identified on video-EEG including: eye blinks, hand or foot movement, chewing and swallowing were also included in the design matrix. SPM [F] maps (p<0.05, corrected for family wise error and p<0.001, uncorrected) were generated for both models. We compared the resulting blood oxygen level dependent (BOLD) maps for cluster size, statistical significance and degree of concordance with the irritative zone. BOLD changes relating to physiological activities were generally seen in expected brain areas. In patients with focal epilepsy, the extent and Z-score of the IED-related global maximum BOLD clusters increased in 4/6 patients and additional IED-related BOLD clusters were observed in 3/6 patients for GLM2. Also, the degree of concordance of IED-related maps with irritative zone improved for one patient for GLM2 and was unchanged for the other cases. In patients with IGE, the size and statistical significance for global maximum and other BOLD clusters increased in 2/4 patients. We conclude that the inclusion of additional regressors, derived from video based information, in the design matrix explains a greater amount of variance and can reveal additional IED-related BOLD clusters which may be part of the epileptic networks.

Reaction Time Correlations during Eye–Hand Coordination: Behavior and Modeling

During coordinated eye-hand movements, saccade reaction times (SRTs) and reach reaction times (RRTs) are correlated in humans and monkeys. Reaction times (RTs) measure the degree of movement preparation and can correlate with movement speed and accuracy. However, RTs can also reflect effector nonspecific influences, such as motivation and arousal. We use a combination of behavioral psychophysics and computational modeling to identify plausible mechanisms for correlations in SRTs and RRTs. To disambiguate nonspecific mechanisms from mechanisms specific to movement coordination, we introduce a dual-task paradigm in which a reach and a saccade are cued with a stimulus onset asynchrony (SOA). We then develop several variants of integrate-to-threshold models of RT, which postulate that responses are initiated when the neural activity encoding effector-specific movement preparation reaches a threshold. The integrator models formalize hypotheses about RT correlations and make predictions for how each RT should vary with SOA. To test these hypotheses, we trained three monkeys to perform the eye-hand SOA task and analyzed their SRTs and RRTs. In all three subjects, RT correlations decreased with increasing SOA duration. Additionally, mean SRT decreased with decreasing SOA, revealing facilitation of saccades with simultaneous reaches, as predicted by the model. These results are not consistent with the predictions of the models with common modulation or common input but are compatible with the predictions of a model with mutual excitation between two effector-specific integrators. We propose that RT correlations are not simply attributable to motivation and arousal and are a signature of coordination.

Independent Allocation of Attention to Eye and Hand Targets in Coordinated Eye-Hand Movements

When reaching for objects, people frequently look where they reach. This raises the question of whether the targets for the eye and hand in concurrent eye and hand movements are selected by a unitary attentional system or by independent mechanisms. We used the deployment of visual attention as an index of the selection of movement targets and asked observers to reach and look to either the same location or separate locations. Results show that during the preparation of coordinated movements, attention is allocated in parallel to the targets of a saccade and a reaching movement. Attentional allocations for the two movements interact synergistically when both are directed to a common goal. Delaying the eye movement delays the attentional shift to the saccade target while leaving attentional deployment to the reach target unaffected. Our findings demonstrate that attentional resources are allocated independently to the targets of eye and hand movements and suggest that the goals for these effectors are selected by separate attentional mechanisms.

The eye dominates in guiding attention during simultaneous eye and hand movements

Prior to the onset of a saccade or a reach, attention is directed to the goal of the upcoming movement. However, it remains unknown whether attentional resources are shared across effectors for simultaneous eye and hand movements. Using a 4-AFC shape discrimination task, we investigated attentional allocation during the planning of a saccade alone, reach alone, or combined saccade and reach to one of five peripheral locations. Target discrimination was better when the probe appeared at the goal of the impending movement than when it appeared elsewhere. However, discrimination performance at the movement goal was not better for combined eye-hand movements compared to either effector alone, suggesting a shared limited attentional resource rather than separate pools of effector-specific attention. To test which effector dominates in guiding attention, we then separated eye and hand movement goals in two conditions: (1) cued reach/fixed saccade--subjects made saccades to the same peripheral location throughout the block, while the reach goal was cued and (2) cued saccade/fixed reach--subjects made reaches to the same location, while the saccade goal was cued. For both conditions, discrimination performance was consistently better at the eye goal than the hand goal. This indicates that shared attentional resources are guided predominantly by the eye during the planning of eye and hand movements.

Preparing coordinated eye and hand movements: Dual-task costs are not attentional

Dual-task costs are observed when people perform two tasks at the same time. It has been suggested that these costs arise from limitations of movement goal selection when multiple goal-directed movements are made simultaneously. To investigate this, we asked participants to reach and look at different locations while we varied the time between the cues to start the eye and the hand movement between 150 ms and 900 ms. In Experiment 1, participants executed the reach first, and the saccade second, in Experiment 2 the order of the movements was reversed. We observed dual-task costs-participants were slower to start the eye or hand movement if they were planning another movement at that time. In Experiment 3, we investigated whether these dual-task costs were due to limited attentional resources needed to select saccade and reach goal locations. We found that the discrimination of a probe improved at both saccade and reach locations, indicating that attention shifted to both movement goals. Importantly, while we again observed the expected dual-task costs as reflected in movement latencies, there was no apparent delay of the associated attention shifts. Our results rule out attentional goal selection as the causal factor leading to the dual-task costs occurring in eye-hand movements.

Attention is predominantly guided by the eye during concurrent eye-hand movements

Attention is predominantly guided by the eye during concurrent eye-hand movements

Modeling eye-hand movement sequences in natural tasks

Modeling eye-hand movement sequences in natural tasks

The use of predictive information is impaired in the actions of children and young adults with Developmental Coordination Disorder

The need for a movement response may often be preceded by some advance information regarding direction or extent. We examined the ability of individuals with Developmental Coordination Disorder (DCD) to organise a movement in response to advance information. Pre-cues were presented and varied in the extent to which they indicated the response target. Both eye movement latencies and hand movements were measured. In the absence of pre-cues, individuals with DCD were as fast in initial hand movements as the typically developing (TD) participants, but were less efficient at correcting initial directional errors. A major difference was seen in the degree to which each group could use advance pre-cue information. TD participants were able to use pre-cue information to refine their actions. For the individuals with DCD this was only effective if there was no ambiguity in the advance cue and they had particular difficulty in using predictive motion cues. There were no differences in the speed of gaze responses which excluded an explanation relating to the dynamic allocation of attention. Individuals with DCD continued to rely on the slower strategy of fixating the target prior to initiating a hand movement, rather than using advance information to set initial movement parameters.

Effects of postural support on eye hand interactions across development

Reaching to grasp an object of interest requires a complex sensorimotor transformation-involving eye, head, hand, and postural systems. We show here that discontinuities in development of movement in these systems are dependent not only on age but also vary according to task constraints. Providing external postural support allows us to examine the differential influences of the eye on the hand and the hand on the eye as the ability to isolate and coordinate each system changes with age. Children 4-6 years old had significant difficulty isolating eye movement from head or hand movement, whereas children 7-9 years old showed improved ability to isolate the eye, and by 10-15 years children became proficient in isolating hand movements from eye movements. Postural support had differential effects on the processes of initiation and execution of eye-hand movements. The addition of postural support decreased the time needed for planning the movement, especially in the youngest children, and contributed to increased speed of isolated movements, whereas it caused differential slowing of coordinated movements depending on the child's developmental level. We suggest that the complexity of the results reflects the complexity of changing task requirements as children transition from simpler ballistic control of all systems to flexible, independent but coordinated control of multiple systems.

Functional dissociation of saccade and hand reaching control with bilateral lesions of the medial wall of the intraparietal sulcus: Implications for optic ataxia

The posterior parietal cortex (PPC) is essential for the integration of visuomotor information during visually guided reaching. Studies in macaque monkeys have demonstrated a functional specialisation around the intraparietal sulcus (IPS) with a more medial representation of hand movements (“parietal reach region”) and a more lateral representation of saccadic eye movements (lateral intraparietal area, LIP). Here we present evidence for the validity of this concept with respect to the human parietal cortex. We recorded isolated and combined goal-directed eye–hand movements in normal control subjects and in a patient with bilateral parieto-occipital lesions and incomplete Balint's syndrome including severe optic ataxia (misreaching to visual targets). Brain lesions in the patient were caused by acute posterior leucoencephalopathy in association with aortic surgery because of Takayasu's arteritis. MRI scans showed bilateral line-shaped hemorrhagic lesions, restricted to the cortex at the medial banks of the intraparietal sulcus, but leaving its lateral banks largely intact. In the patient visually guided reaching was significantly dysmetric, whereas the metrics of visually guided saccades were within normal limits. Dysmetria was more pronounced for the right visual field, with a gross hypermetria. Variability of the movement improved when a delay of 5 or 10 s was introduced between target presentation and movement execution. Lesion data support the concept of a functional specialisation around the human IPS: The cortex medial to the IPS predominantly controls rapid goal-directed reaching movements, comparable to the parietal reach region in monkeys, whereas saccadic eye movements appear to be controlled rather by the cortex lateral to the IPS.

Temporal Evolution and Strength of Neural Activity in Parietal Cortex during Eye and Hand Movements

The role of area 7a in eye-hand movement was studied by recording from individual neurons while monkeys performed 7 different tasks, aimed at assessing the relative influence of retinal, eye, and hand information on neural activity. Parietal cell activity was modulated by visuospatial signals about target location, as well as by information concerning eye and/or hand movement, and position. The highest activity was elicited when the hand moved to the fixation point. The population activities across different memory tasks showed common temporal peaks when aligned to the visual instruction (visuospatial peak) or Go signal (motor peak) for eye, hand, and coordinated eye-hand movement. The motor peak was higher for coordinated eye-hand movement, and it was absent in a No-Go task. Two activation maxima were also observed during visual reaching. They had the same latency of the visuospatial and motor peaks seen in the memory tasks. Therefore, area 7a seems to operate through a common neural mechanism underlying eye, hand, or combined eye-hand movement. This mechanism is revealed by invariant temporal activity profiles and is independent from the effector selected and from the presence or absence of a visible target during movement. For comparative purposes, we have studied the temporal evolution of the population activity in the superior parietal lobule (SPL) during the same reaching tasks and during a saccade task. In SPL, the population activity was characterized by a single peak, time locked to the Go signal for eye, hand, or combined eye-hand movement. As in IPL, the time of occurrence of this peak was effector independent. The population activity remained unchanged when the position of the eye changed, suggesting that SPL is mostly devoted to the hand motor behavior.

Eye–hand coordination in essential tremor

Patients with essential tremor (ET) or with cerebellar lesions have in common oculomotor abnormalities, with the exception of saccadic eye movements, which do not seem to be involved in ET. Since grasping is prolonged in ET and might be related to saccadic dysmetria, we tested whether simultaneous hand pointing could unmask it. Twelve ET patients and 14 controls performed saccades with and without simultaneous pointing movements to the same targets, and with and without a gap between the disappearance of the fixation point and the appearance of the target. Eye movements were recorded with the magnetic search-coil method, hand movements with an ultrasound-emitting probe. ET patients did not have saccadic dysmetria, and contrary to normal subjects their saccadic latency did not decrease during combined eye-hand movements compared with saccades performed in isolation. Hand movements had a longer duration in ET patients, with decreased peak acceleration, an increased latency of the peak velocity, and peak deceleration. In conclusion, this first study on eye-hand coordination in ET revealed abnormal kinematic changes in the early phase of pointing movements. These changes might be related to cerebellar disease but they are independent of the intention tremor component and saccade performance.

Preserved eye-hand coordination in spinocerebellar ataxia type 17

Background/Aim: SCA17 is caused by a trinucleotide expansion in the gene encoding the TATA binding protein (TBP). We intended to characterize combined eye-hand movements in this ataxic disorder by using hand pointing tasks.

Where the Eye Looks, the Hand FollowsLimb-Dependent Magnetic Misreaching in Optic Ataxia

The posterior parietal cortex (PPC) is thought to play an important role in the sensorimotor transformations associated with reaching movements. In humans, damage to the PPC, particularly bilateral lesions, leads to impairments of visually guided reaching movements (optic ataxia). Recent accounts of optic ataxia based upon electrophysiological recordings in monkeys have proposed that this disorder arises because of a breakdown in the tuning fields of parietal neurons responsible for integrating spatially congruent retinal, eye, and hand position signals to produce coordinated eye and hand movements [1Battaglia-Mayer A. Caminiti R. Optic ataxia as a result of the breakdown of the global tuning fields of parietal neurones.Brain. 2002; 125: 225-237Crossref PubMed Scopus (137) Google Scholar]. We present neurological evidence that forces a reconceptualization of this view. We report a detailed case study of a patient with a limb-dependent form of optic ataxia who can accurately reach with either hand to objects that he can foveate (thereby demonstrating coordinated eye-hand movements) but who cannot effectively decouple reach direction from gaze direction for movements executed using his right arm. The demonstration that our patient's misreaching is confined to movements executed using his right limb, and only for movements that are directed to nonfoveal targets, rules out explanations based upon simple perceptual or motor deficits but indicates an impairment in the ability to dissociate the eye and limb visuomotor systems when appropriate.

Where the Eye Looks, the Hand Follows

The posterior parietal cortex (PPC) is thought to play an important role in the sensorimotor transformations associated with reaching movements. In humans, damage to the PPC, particularly bilateral lesions, leads to impairments of visually guided reaching movements (optic ataxia). Recent accounts of optic ataxia based upon electrophysiological recordings in monkeys have proposed that this disorder arises because of a breakdown in the tuning fields of parietal neurons responsible for integrating spatially congruent retinal, eye, and hand position signals to produce coordinated eye and hand movements . We present neurological evidence that forces a reconceptualization of this view. We report a detailed case study of a patient with a limb-dependent form of optic ataxia who can accurately reach with either hand to objects that he can foveate (thereby demonstrating coordinated eye-hand movements) but who cannot effectively decouple reach direction from gaze direction for movements executed using his right arm. The demonstration that our patient's misreaching is confined to movements executed using his right limb, and only for movements that are directed to nonfoveal targets, rules out explanations based upon simple perceptual or motor deficits but indicates an impairment in the ability to dissociate the eye and limb visuomotor systems when appropriate.