Visual Error Signals Research Articles

Cerebellar lesions and prism adaptation in macaque monkeys.

If a laterally displacing prism is placed in front of one eye of a person or monkey with the other eye occluded, they initially will point to one side of a target that is located directly in front of them. Normally, people and monkeys adapt easily to the displaced vision and correct their aim after a few trials. If the prism then is removed, there is a postadaptation shift in which the subject misses the target and points in the opposite direction for a few trials. We tested five Macaque monkeys for their ability to adapt to a laterally displacing prism and to show the expected postadaptation shift. When tested as normals, all five animals showed the typical pattern of adaptation and postadaptation shift. Like human subjects, the monkeys also showed complete interocular transfer of the adaptation but no transfer of the adaptation between the two arms. When preoperative training and testing was complete, we made lesions of various target areas on the cerebellar cortex. A cerebellar lesion that included the dorsal paraflocculus and uvula abolished completely the normal prism adaptation for the arm ipsilateral to the lesion in one of the five monkeys. The other four animals retained the ability to prism-adapt normally and showed the expected postadaptation shift. In the one case in which the lesion abolished prism adaptation, the damage included Crus I and II, paramedian lobule and the dorsal paraflocculus of the cerebellar hemispheres as well as lobule IX, of the vermis. Thus in this case, the lesion included virtually all the cerebellar cortex that receives mossy-fiber visual information relayed via the pontine nuclei from the cerebral cortex. The other four animals had damage to lobule V, the classical anterior lobe arm area and/or vermian lobules VI/VII, the oculomotor region. When tested postoperatively, some of these animals showed a degree of ataxia equivalent to that of the case in which prism adaptation was affected, but prism adaptation and the postadaptation shift remained normal. We conclude that in addition to its role in long-term motor learning and reflex adaptation, the region of the cerebellum that was ablated also may be a critical site for a short-term motor memory. Prism adaptation seems to involve a region of the cerebellum that receives a mossy-fiber visual error signal and probably a corollary discharge of the movement.

The neural processing of 3‐D visual information: evidence from eye movements

Primates have several reflexes that generate eye movements to compensate for bodily movements that would otherwise disturb their gaze and undermine their ability to process visual information. Two vestibulo-ocular reflexes compensate selectively for rotational and translational disturbances of the head, and each has visual backups that operate as negative feedback tracking mechanisms to deal with any residual disturbances of gaze. Of particular interest here are three recently discovered visual tracking mechanisms that specifically address translational disturbances and operate in machine-like fashion with ultra-short latencies (< 60 ms in monkeys, < 85 ms in humans). These visual reflexes deal with motions in all three dimensions and operate as automatic servos, using preattentive parallel processing to provide signals that initiate eye movements before the observer is even aware that there has been a disturbance. This processing is accomplished by visual filters each tuned to a different feature of the binocular images located in the immediate vicinity of the plane of fixation. Two of the reflexes use binocular stereo cues and the third is tuned to particular patterns of optic flow associated with the observer's forward motion. Some stereoanomalous subjects show tracking deficits that can be attributed to a lack of just one subtype of cortical cell encoding motion in one particular direction in a narrow depth plane centred on fixation. Despite their rapid, reflex nature, all three mechanisms rely on cortical processing and evidence from monkeys supports the hypothesis that all are mediated by the medial superior temporal (MST) area of cortex. Remarkably, MST seems to represent the first stage in cortical motion processing at which the visual error signals driving each of the three reflexes are fully elaborated at the level of individual cells.

Short-term adaptation of electrically induced saccades in monkey superior colliculus.

1. This study focuses on the neural mechanisms underlying short-term adaptation of saccadic eye movements in the rhesus monkey. Involuntary saccades of various amplitudes and directions (E-saccades) were elicited in complete darkness by electrical stimulation (< or = 50 microA) in the deeper layers of the superior colliculus (SC) at 30 different sites in two monkeys. E-saccades at a given site could be adapted by presenting a visual target at a small distance from the expected end point immediately after their occurrence. The monkeys were trained to null the ensuing error signal by making the appropriate correction saccade to the visual target in many successive trials (E-adap paradigm). By properly adjusting the location of the visual target relative to the end point of the E-saccade, the latter could be modified in amplitude as well as in direction. 2. E-saccade modifications were highly significant, always in the intended direction, and occurred only if a postsaccadic visual error signal was created. These changes were plastic and required a subsequent E-adap series with an opposite error signal to cancel them. Their time course, both during the adaptation and the readaptation period, indicated that the modification was a slow and gradual process, as has been observed earlier in classical visual adaptation experiments. 3. Postadaptation tests, assessing whether the adaptation of E-saccades was also noticeable in normal visually guided saccades (V-saccades), showed incomplete adaptation transfer that was significant in most cases. A similar result, significant in all cases, was obtained with an extended version of the E-adap paradigm in which movement planning on the basis of target selection was possible. In this case, a presaccadic visual target was presented at the expected end point of the E-saccade, which was evoked just before the monkey would make a voluntary saccade itself (VE-adap). 4. In another series of experiments, V-saccades, which were matched to the optimal saccade vector of the particular collicular site under investigation, were adapted with the classical intrasaccadic target shift paradigm (V-adap). In agreement with earlier findings, this V-adaptation showed no transfer to the E-saccades. This result was obtained even in trials in which movement planning on the basis of target selection was possible (VE-test). 5. Our experiments have shown that saccades of collicular origin can be adapted and that presaccadic target selection is not crucial for this process. Both results are nicely in line with an existing model featuring a downstream adaptive corrector with access to SC inputs. This scheme, however, does not explain why the degree of saccadic adaptation, achieved by applying any of the three adaptation paradigms (E-adap, EV-adap, or V-adap), was never equally expressed in V- and E-saccades. Arguments for extending the model by adding a cortical input from the frontal eye fields to the adaptive corrector are discussed.

Some observations on abduction nystagmus in internuclear ophthalmoplegia

The findings in 220 consecutive patients with internuclear ophthalmoplegia (INO) investigated by direct-current electro-oculography (EOG) are reported. Abduction nystagmus is significantly more frequent in patients with severe INO and increases with increasing lateral movements in the direction of the paretic medial rectus muscle. In a series of 12 unselected patients, additional recordings were made immediately after patching the paretic eye, and in six of them also immediately after patching the non-pa-retic eye. Patching the paretic eye was followed by a decreased pulse-step mismatch and a decreased pulse duration causing decreased amplitudes of abduction and adduction saccades in most patients. Patching the non-paretic eye was followed by an increased pulse-step mismatch and an increased pulse duration causing increased amplitudes of abduction and adduction saccades in most patients. The author's findings support the adaptation hypothesis of abduction nystagmus in most patients with INO, and are in accordance with Hering's law of equal innervation. The increased innervation is adjusted to the adduction paresis and generated immediately after onset of the paresis. Abduction nystagmus in INO depends on the medial rectus paresis and presumably is initiated by a visual error signal.

Effects of delayed visual information on the rate and amount of prism adaptation in the human

Accurate reaching towards a visual target is initially disturbed when the visual field is displaced by prisms, but recovers with successive trials. To determine how the improvement depends on the visual error signals associated with the motor output, the time course of prism adaptation was studied with delayed visual information on the error. Subjects were trained to reach rapidly at a target on a tangent screen. Vision was always blocked during the movement, and allowed again only after the index finger touched the screen. One experiment consisted of three sets of 30 trials. In the first set, the subject wore no prisms and vision was allowed without delay. In the second, the visual field was displaced by prisms, and vision was available only after a delay period of 0-10,000 msec while the subjects maintained their final pointing position. Initially, the subject misreached the target by about the amount of visual displacement (60 mm). Errors decreased with trials by an amount proportional to the error in the preceding trial. The rate of decrease of error was generally largest when the delay was 0 msec, became significantly smaller when the delay was 50 msec, and showed only gradual change with longer delays. In the third set, the subject wore no prisms and vision was allowed without delay. Initial misreaching in the direction opposite to the visual displacement, reflecting the amount of adaptation in the second set, was generally largest with no delay (median of 46 mm) and significantly smaller with 50 msec and longer delays (17-33 mm).(ABSTRACT TRUNCATED AT 250 WORDS)

Relationship of calculating the Jacobian matrices of nonlinear systems and population coding algorithms in neurobiology

In this paper we examine how the calculation of Jacobian matrices in nonlinear systems is related to population coding algorithms in neurobiology. Population coding algorithms have been designed to retrodict sensory stimuli or predict motor behavior from neuronal responses. We simulate the visuomotor hand movement task of reaching in a plane. Adaptive feedback control updating of joint angles of a three-joint arm was used in the model. The control signal is the dot product between the visual error signal and the Jacobian matrix of the direct kinematic equation of hand movement. The x and y hand trajectories can follow the x and y time series of Lorenz and Ro¨ssler systems of coupled nonlinear equations. In this particular example, the elements of the Jacobian matrix can be estimated from observed changes in joint angles and Cartesian coordinate values of the hand. We also point out how the Jacobian matrices in nonlinear biological systems can be estimated in general from observed time series; e.g. joint angles, Cartesian coordinate values of hand movement, neural or muscle activity. Estimating Jacobian matrices of nonlinear functions from experimental observations can provide not only data analysis but also signal synthesis and can lead to a more realistic modeling of sensorimotor transformations.

Adaptive vector integration to endpoint: Self-organizing neural circuits for control of planned movement trajectories

A neural network model is described for adaptive control of arm movement trajectories during visually guided reaching. The model clarifies how a child, or infant robot, can learn to reach for objects that it sees. Piaget has provided basic insights with his concept of a circular reaction. As an infant makes internally generated hand movements, the eyes automatically follow this motion. A transformation is learned between the visual representation of hand position and the motor representation of hand position. Learning of this transformation eventually enables the child to accurately reach for visually detected targets. Grossberg and Kuperstein (1989) have shown how the eye movement system can use visual error signals to correct movement parameters via cerebellar learning. Here it is shown how the arm movement system can endogenously generate movements which lead to adaptive tuning of arm control parameters. These movements also activate the target position representations that are used to learn the visou-motor transformation that controls visually guided reaching. The arm movement properties obtain in the Adaptive Vector Integration to Endpoint (AVITE) model an adaptive neural circuit based on the VITE model for arm and speech trajectory generation of Bullock and Grossberg (1988a).

Vector associative maps: Unsupervised real-time error-based learning and control of movement trajectories

This article described neural network models for adaptive control of arm movement trajectories during visually guided reaching and, more generally, a framework for unsupervised real-time error-based learning. The models clarify how a child, or untrained robot, can learn to reach for objects that it sees. Piaget has provided basic insights with his concept of circular reaction: As an infant makes internally generated movements of its hand, the eyes automatically follow this motion. A transformation is learned between the visual representation of hand position and the motor representation of hand position. Learning of this transformation eventually enables the child to accurately reach for visually detected targets. It has been shown how the eye movement system can use visual error signals to correct movement parameters via cerebellar learning. Here it is shown how endogenously generated arm movements lead to adaptive tuning of arm control parameters. These movements also activate the target position representations that are used to learn the visuo-motor transformation that controls visually guided reaching. The AVITE model presented here is an adaptive neural circuit based on the Vector Integration to Endpoint (VITE) model for arm and speech trajectory generation of Bullock and Grossberg. In the VITE model, a Target Position Command (TPC) represents the location of the desired target. The Present Position Command (PPC) encodes the present hand-arm configuration. The Difference Vector (DV) population continuously computes the difference between the PPC and TPC. A speed-controlling GO signal multiplies DV output. The PPC integrates the (DV) · (GO) product and generates an outflw command to the arm. Integration at the PPC continues at a rate dependent on GO signal size until the DV reaches zero, at which time the PPC equals the TPC. The AVITE model explains how self-consistent TPC and PPC coordinates are autonomously generated and learned. Learning of AVITE parameters is regulated by activation of a self-regulating Endogenous Random Generator (ERG) of training vectors. Each vector is integrated at the PPC, giving rise to a movement command. The generation of each vector induces a complementary postural phase during which ERG output stops and learning occurs. Then a new vector is generated and the cycle is repeated. This cyclic, biphasic behavior is controlled by a specialized gated dipole circuit. ERG output autonomously stops in such a way that, across trials, a broad sample of workspace target positions is generated. When the ERG shuts off, a modulator gate opens, copying the PPC into the TPC. Learning of a transformation from TPC to PPC occurs using the DV as an error signal that is zeroed due to learning. This learning scheme is called a Vector Associative Map, or VAM. The VAM model is a general-purpose device for autonomous real-time-error based learning and performance of associative maps. The DV stage serves the dual function of reading out new TPCs during performance and reading in new adaptive weights during learning, without a disruption of real-time operation. VAMs thus provide an on-line unsupervised alternative to the off-line properties of supervised error-correction learning algorithms. VAMs and VAM cascades for learning motor-to-motor and spatial-to-motor maps are described. VAM models and Adaptive Resonance Theory (ART) models exhibit complementary matching, learning, and performance properties that together provide a foundation for designing a total sensory-cognitive and cognitive-motor autonomous system.

A vector-sum process produces curved aiming paths under rotated visual-motor mappings

Under a 90 degrees rotation of motor space relative to visual space, human two-dimensional aiming movements frequently take the form of smooth arcs such as spirals and semi-circles. A time-independent differential equation explains this tendency in terms of a rotation-induced vector field made up, at each point in the two-dimensional space, of two input vectors. One vector represents a visual error signal and the other represents a motor error signal. A trajectory's instantaneous direction of movement at each point can be described as the resultant of the two vectors. This mathematical formulation incorporates plausible visual-motor mechanisms and, when expressed in polar coordinates, leads to a new method for analyzing the spatial properties of movements (i.e., movement paths). Plots of the angle between the resultant and the target vector (phi) against distance from the target (r, in the polar representation) summarize the arc-shaped movement paths as a simple relation that can be analyzed statistically with respect to properties such as monotonicity. The polar representation is a plausible representation of visually-guided movements, with the visual error vector functioning as an objective function relative to which behavior is optimized. We extend the model and the r, phi movement path analysis to non-90 degrees rotations, and we find that the model predicts an observed qualitative shift in behavior for rotations greater than 90 degrees. It also predicts qualitatively different path shapes observed under visual-motor reflections.

Visual control of steering in locust flight: the effects of head movement on responses to roll stimuli

The contribution of head movement to the control of roll responses in flying locusts (Locusta migratoria) has been examined (i) on a flight balance, recording the angles through which the locust turns when following an artificial horizon; (ii) by recording activity in a pair of flight muscles in restrained conditions; and (iii) by observations on free flying locusts. Responses were compared when the head was free to turn about the thorax, as normal, and when the head was waxed to the thorax, blocking any relative motion between the two (‘head-fixed’). These experiments suggest that the major signal generating corrective roll manoeuvres is the visual error between the angle of the head and the horizon, rather than a signal that includes a measure of the head-thorax angle. 1. On the flight balance in the head-free condition the roll angle of the thorax was consistently less than in the head-fixed state, and followed the stimulus with longer response lags. Furthermore, the difference between the angle of the thorax assumed during head-free and head-fixed rolls was close to the angle of the head relative to the thorax during head-free responses. 2. Records of activity of the forewing first basalar muscles (M97) were made during rotation of the horizon about immobilized animals. When the head could follow the horizon, the relative latency between activity in the left and right basalar muscles decreased as the head position turned to approach the displaced horizon. When head-fixed, the relative latency was directly proportional to horizon angle. 3. The relative latency between left and right M97 flight muscles correlates better with the visual error signal than with the horizon position signal, lagging by approximately 40 ms. 4. In the open air, head-fixed locusts appear able to fly as well as head-free locusts.

Planning sequences of saccades

Subjects used saccades to fixate a sequence of 1–5 stationary targets (separation= 90′) located at the vertices of an imaginary pentagon. The latency of the first saccade in a sequence and the duration of intervals between subsequent saccades increased with sequence length at a rate of about 20 msec/target. Latency also varied with ordinal position in the sequence. These results were not due to directional differences in saccadic latency nor to latency-accuracy or to latency-precision trade-offs. Results were similar when targets were removed and saccades were directed to remembered locations. These effects may be best accounted for by models that have been proposed to account for similar effects of sequence length and ordinal position on other voluntary motor tasks, such as typing, speech and finger-tapping. In these models motor programs for a sequence of responses are planned before execution and then retrieved from memory during execution. These models are fundamentally different from the traditional saccadic models in which visual error signals evoke saccades. Instead, we propose that saccades are controlled by an organized plan for an entire sequence of saccades. Visual error signals may modify or elaborate the plans during the execution of a sequence. Our proposal is consistent with ideas developed by Lashley [ Cerebral Mechanisms in Behavior: The Hixon Symposium. Wiley, New York (1951)] in his general treatment of the central organization that determines voluntary motor performance.

Test of the ballistic hypothesis of movement and the ‘point of no return’

How long does it take to stop or modify a movement once it has begun? Where there is an error in producing an action as intended (usually because of control-display incompatibility) subjects may correct it in less than their normal reaction time (RT) by proprioception1 or efference-copy monitoring2—4. Where subjects work from a visual (external) error signal, however, they are usually said to take a full RT to stop5 or correct6,7 ongoing movements, so that there is a ‘point of no return’, after which they are committed for about 300 ms to any action decided on8. During this time the action is ballistic at the perceptual-motor level6,9,10. But in a recent tracking experiment11 where subjects wrongly anticipated the target motion, and so began a movement in the wrong direction, the correction time to stop the action (CorT) was often much shorter than the normal RT to initiate movements, down to as little as 120 ms in some cases. This suggests that fast error correction can occur from a visual signal alone, at least to stop movements (as in an emergency). Repeating the experiment we find that movements can indeed be arrested soon after they begin, but only if the beginning is delayed; it always takes one RT from the occurrence of the error signal. We therefore infer that there is a ‘point of no return’ at the decision level, but that movements are not necessarily ballistic at the motor level.

Different programming modes of human saccadic eye movements as a function of stimulus eccentricity: indications of a functional subdivision of the visual field.

1. Voluntary saccadic eye movements were made toward flashes of light on the horizontal meridian, whose duration and distance from the point of fixation were varied; eye movements were measured using d.c.-electrooculography.--2. Targets within 10°---15° eccentricity are usually reached by one saccadic eye movement. When the eyes turn toward targets of more than 10°---15° eccentricity, the first saccadic eye movement falls short of the target by an angle usually not exceeding 10°. The presence of the image of the target off the fovea (visual error signal) subsequent to such an undershoot elicits, after a short interval, corrective saccades (usually one) which place the image of the target on the fovea. In the absence of a visual error signal, the probability of occurrence of corrective saccades is low, but it increases with greater target eccentricities. These observations suggest that there are different, eccentricity-dependent modes of programming saccadic eye movements.--3. Saccadic eye movements appear to be programmed in retinal coordinates. This conclusion is based on the observations that, irrespective of the initial position of the eyes in the orbit, a) there are different programming modes for eye movements to targets within and beyond 10°---15° from the fixation point, and b_ the maximum velocity of saccadic eye movements is always reached at 25° to 30° target eccentricity. --4. Distributions of latency and intersaccadic interval (ISI) are frequently multimodal, with a separation between modes of 30 to 40 msec. These observations suggest that saccadic eye movements are produced by mechanisms which, at a frequency of 30 Hz, process visual information. --5. Corrective saccades may occur after extremely short intervals (30 to 60 msec) regardless of whether or not a visual error signal is present; the eyes may not even come to a complete stop during these very short intersaccadic intervals. It is suggested that these corrective saccades are triggered by errors in the programming of the initial saccadic eye movements, and not by a visual error signal. --6. The exitence of different, eccentricity-dependent programming modes of saccadic eye movements, is further supported by anatomical, physiological, psychophysical, and neuropathological observations that suggest a dissociation of visual functions dependent on retinal eccentricity. Saccadic eye movements to targets more eccentric than 10°---15° appear to be executed by a mechanism involving the superior colliculus (perhaps independent of the visual cortex), whereas saccadic eye movements to less eccentric targets appear to depend on a mechanism involving the geniculo-cortical pathway (perhaps in collaboration with the superior colliculus).

The smallest voluntary saccade: Implications for fixation

(1) Subjects can make voluntary saccades as small as fixation saccades if the target is visible (mean size < 5.7′ arc S.D. < 3.0). These saccades were made away from the preferred fixation position showing that a visual error signal is not necessary, but a visible target is because voluntary saccades were 3–4 times as large when made after the target was removed from view. (2) Subjects do make some spontaneous saccades, but very rarely throughout 7 sec trials when explicitly instructed to suppress them (1 every 25 sec). Such saccades were large relative to voluntary and typical fixation saccades (mean size > 10.0′ arc). (3) Subjects were usually aware of making spontaneous saccades (90 per cent detections), but were better able to detect and identify the direction of target steps than their own spontaneous saccades. These results are compatible with a visual search interpretation of the saccadic component of the maintained fixation pattern.