Visual Motion Signals Research Articles

Integrated Perceptual Decisions Rely on Parallel Evidence Accumulation.

The ability to make accurate and timely decisions, such as judging when it is safe to cross the road, is the foundation of adaptive behavior. While the computational and neural processes supporting simple decisions on isolated stimuli have been well characterized, decision-making in the real world often requires integration of discrete sensory events over time and space. Most previous experimental work on perceptual decision-making has focused on tasks that involve only a single, task-relevant source of sensory input. It remains unclear, therefore, how such integrative decisions are regulated computationally. Here we used psychophysics, electroencephalography, and computational modeling to understand how the human brain combines visual motion signals across space in the service of a single, integrated decision. To that purpose, we presented two random-dot kinematograms in the left and the right visual hemifields. Coherent motion signals were shown briefly and concurrently in each location, and healthy adult human participants of both sexes reported the average of the two motion signals. We directly tested competing predictions arising from influential serial and parallel accounts of visual processing. Using a biologically plausible model of motion filtering, we found evidence in favor of parallel integration as the fundamental computational mechanism regulating integrated perceptual decisions.

Illusory motion and vection induced by a printed static image under flickering ambient light at rates up to 100 Hz.

Visual motion signals can produce self-motion perception known as vection in observers. Vection can be generated by illusory motions in the form of global expantion in still images as well as by visual motion signals. The perception of vection can be enhanced by flickering images at a rate of 5 Hz. This study examined the illusory motion and vection induced by a printed static image under flickering ambient light at rates up to 100 Hz. The perception of illusory motion and vection were enhanced by flickering ambient lights at 50, 75, and 100 Hz. The enhancement effect was higher for the flicker rates expected to be detectable by humans. The findings of this study suggest that alternating bright and dark signals to the cone receptors and primary visual cortex trigger perceptions of illusory motions.

Incongruent active head rotations increase visual motion detection thresholds.

Attributing a visual motion signal to its correct source-be that external object motion, self-motion, or some combination of both-seems effortless, and yet often involves disentangling a complex web of motion signals. Existing literature focuses on either translational motion (heading) or eye movements, leaving much to be learnt about the influence of a wider range of self-motions, such as active head rotations, on visual motion perception. This study investigated how active head rotations affect visual motion detection thresholds, comparing conditions where visual motion and head-turn direction were either congruent or incongruent. Participants judged the direction of a visual motion stimulus while rotating their head or remaining stationary, using a fixation-locked Virtual Reality display with integrated head-movement recordings. Thresholds to perceive visual motion were higher in both active-head rotation conditions compared to stationary, though no differences were found between congruent or incongruent conditions. Participants also showed a significant bias to report seeing visual motion travelling in the same direction as the head rotation. Together, these results demonstrate active head rotations increase visual motion perceptual thresholds, particularly in cases of incongruent visual and active vestibular stimulation.

An Automated Skill Assessment Framework Based on Visual Motion Signals and a Deep Neural Network in Robot-Assisted Minimally Invasive Surgery.

Surgical skill assessment can quantify the quality of the surgical operation via the motion state of the surgical instrument tip (SIT), which is considered one of the effective primary means by which to improve the accuracy of surgical operation. Traditional methods have displayed promising results in skill assessment. However, this success is predicated on the SIT sensors, making these approaches impractical when employing the minimally invasive surgical robot with such a tiny end size. To address the assessment issue regarding the operation quality of robot-assisted minimally invasive surgery (RAMIS), this paper proposes a new automatic framework for assessing surgical skills based on visual motion tracking and deep learning. The new method innovatively combines vision and kinematics. The kernel correlation filter (KCF) is introduced in order to obtain the key motion signals of the SIT and classify them by using the residual neural network (ResNet), realizing automated skill assessment in RAMIS. To verify its effectiveness and accuracy, the proposed method is applied to the public minimally invasive surgical robot dataset, the JIGSAWS. The results show that the method based on visual motion tracking technology and a deep neural network model can effectively and accurately assess the skill of robot-assisted surgery in near real-time. In a fairly short computational processing time of 3 to 5 s, the average accuracy of the assessment method is 92.04% and 84.80% in distinguishing two and three skill levels. This study makes an important contribution to the safe and high-quality development of RAMIS.

Motion vision: Fish swimming to see

Self-motion generates optic flow, a visual motion signal used by many organisms for navigation and self-stabilization. A new study quantitatively demonstrates how environmental structure and current behavioral state explain the spatial biases observed in zebrafish optomotor responses.

Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila.

Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.

Involvement of visual signals in kinaesthesia: A virtual reality study

Body movements are invariably accompanied by various proprioceptive, visual, tactile and/or motor signals. It is therefore difficult to completely dissociate these various signals from each other in order to study their specific involvement in the perception of movement (kinaesthesia). Here, we manipulated visual motion signals in a virtual reality display by using a humanoid avatar. The visual signals of movement could therefore be manipulated freely, relative to the participant’s actual movement or lack of movement. After an embodiment phase in which the avatar’s movements were coupled to the participant’s voluntary movements, kinaesthetic illusions were evoked by moving the avatar’s right forearm (flexion or extension) while the participant’s right arm remained static. The avatar’s left forearm was hidden from view. In parallel, somaesthetic signals could be masked by agonist–antagonist co-vibration or be amplified (by agonist vibration only or antagonist vibration only) so that the real impact of visual cues of movement in kinaesthesia could be studied. In a study of 24 participants, masking the somaesthetic signals (which otherwise provide signals indicating that the arm is static) was associated with a greater intensity and shorter latency of the visually evoked illusions. These results confirm the importance of carefully considering somaesthetic signals when assessing the contribution of vision to kinaesthesia. The use of a combination of virtual reality and somaesthetic signal manipulation might be of clinical value.

Integration of visual motion and orientation signals in dyslexic children: an equivalent noise approach.

Dyslexic individuals have been reported to have reduced global motion sensitivity, which could be attributed to various causes including atypical magnocellular or dorsal stream function, impaired spatial integration, increased internal noise and/or reduced external noise exclusion. Here, we applied an equivalent noise experimental paradigm alongside a traditional motion-coherence task to determine what limits global motion processing in dyslexia. We also presented static analogues of the motion tasks (orientation tasks) to investigate whether perceptual differences in dyslexia were restricted to motion processing. We compared the performance of 48 dyslexic and 48 typically developing children aged 8 to 14 years in these tasks and used equivalent noise modelling to estimate levels of internal noise (the precision associated with estimating each element's direction/orientation) and sampling (the effective number of samples integrated to judge the overall direction/orientation). While group differences were subtle, dyslexic children had significantly higher internal noise estimates for motion discrimination, and higher orientation-coherence thresholds, than typical children. Thus, while perceptual differences in dyslexia do not appear to be restricted to motion tasks, motion and orientation processing seem to be affected differently. The pattern of results also differs from that previously reported in autistic children, suggesting perceptual processing differences are condition-specific.

Common and independent processing of visual motion perception and oculomotor response.

Visual motion signals are used not only to drive motion perception but also to elicit oculomotor responses. A fundamental question is whether perceptual and oculomotor processing of motion signals shares a common mechanism. This study aimed to address this question using visual motion priming, in which the perceived direction of a directionally ambiguous stimulus is biased in the same (positive priming) or opposite (negative priming) direction as that of a priming stimulus. The priming effect depends on the duration of the priming stimulus. It is assumed that positive and negative priming are mediated by high- and low-level motion systems, respectively. Participants were asked to judge the perceived direction of a π-phase-shifted test grating after a smoothly drifting priming grating during varied durations. Their eye movements were measured while the test grating was presented. The perception and eye movements were discrepant under positive priming and correlated under negative priming on a trial-by-trial basis when an interstimulus interval was inserted between the priming and test stimuli, indicating that the eye movements were evoked by the test stimulus per se. These findings suggest that perceptual and oculomotor responses are induced by a common mechanism at a low level of motion processing but by independent mechanisms at a high level of motion processing.

Aerial course stabilization is impaired in motion-blind flies.

Visual motion detection is among the best understood neuronal computations. As extensively investigated in tethered flies, visual motion signals are assumed to be crucial to detect and counteract involuntary course deviations. During free flight, however, course changes are also signalled by other sensory systems. Therefore, it is as yet unclear to what extent motion vision contributes to course control. To address this question, we genetically rendered flies motion-blind by blocking their primary motion-sensitive neurons and quantified their free-flight performance. We found that such flies have difficulty maintaining a straight flight trajectory, much like unimpaired flies in the dark. By unilateral wing clipping, we generated an asymmetry in propulsive force and tested the ability of flies to compensate for this perturbation. While wild-type flies showed a remarkable level of compensation, motion-blind animals exhibited pronounced circling behaviour. Our results therefore directly confirm that motion vision is necessary to fly straight under realistic conditions.

The cingulate oculomotor cortex

Knowledge about the relevance and extent of human eye movement control in the cingulate cortex to date is very limited. Experiments in non-human primates brought about evidence for a potentially central role of the dorsal bank of the cingulate sulcus in saccadic eye movements. In humans, a putative cingulate eye field (CEF) in the same region has been proposed; however, its function and location still remain controversial. Another area in the posterior cingulate cortex, the cingulate sulcus visual area (CSv), has been shown to respond to visual motion cues and also ocular motor tasks. In this study we used multi-band neuroimaging (n = 46) to comprehensively characterize oculomotor responses along the entire cingulate cortex during the most common types of eye movements.We were able to robustly localize the CEF to the anterior portion of the midcingulate gyrus. The region gave responses during all oculomotor tasks and is embedded within the ventral attention network. Area CSv, which is located in the anterior portion of the posterior cingulate gyrus, on the other hand responded to smooth pursuit and optokinetic nystagmus only. It likewise represents a node within the ventral attention network but at the same time seems to be a distinctive part of the somatomotor network. Our findings support an executive role of the CEF, suggesting a cognitive control function in maintaining and adapting different kinds of eye movements. CSv on the other hand might be an interface for relaying oculomotor, visual motion and broad sensory signals related to self-motion.

Seeing motion of controlled object improves grip timing in adults with autism spectrum condition: evidence for use of inverse dynamics in motor control.

Previous studies (Haswell et al. in Nat Neurosci 12:970-972, 2009; Marko et al. in Brain J Neurol 138:784-797, 2015) reported that people with autism rely less on vision for learning to reach in a force field. This suggested a possibility that they have difficulties in extracting force information from visual motion signals, a process called inverse dynamics computation. Our recent study (Takamuku et al. in J Int Soc Autism Res 11:1062-1075, 2018) examined the ability of inverse computation with two perceptual tasks and found similar performances in typical and autistic adults. However, this tested the computation only in the context of sensory perception while it was possible that the suspected disability is specific to the motor domain. Here, in order to address the concern, we tested the use of inverse dynamics computation in the context of motor control by measuring changes in grip timing caused by seeing/not seeing a controlled object. The motion of the object was informative of its inertial force and typical participants improved their grip timing based on the visual feedback. Our interest was on whether the autism participants show the same improvement. While some autism participants showed atypical hand slowing when seeing the controlled object, we found no evidence of abnormalities in the inverse computation in our grip timing task or in a replication of the perceptual task. This suggests that the ability of inverse dynamics computation is preserved not only for sensory perception but also for motor control in adults with autism.

Different mechanisms for modulation of the initiation and steady-state of smooth pursuit eye movements.

Smooth pursuit eye movements are used by primates to track moving objects. They are initiated by sensory estimates of target speed represented in the middle temporal (MT) area of extrastriate visual cortex and then supported by motor feedback to maintain steady-state eye speed at target speed. Here, we show that reducing the coherence in a patch of dots for a tracking target degrades the eye speed both at the initiation of pursuit and during steady-state tracking, when eye speed reaches an asymptote well below target speed. The deficits are quantitatively different between the motor-supported steady-state of pursuit and the sensory-driven initiation of pursuit, suggesting separate mechanisms. The deficit in visually guided pursuit initiation could not explain the deficit in steady-state tracking. Pulses of target speed during steady-state tracking revealed lower sensitivities to image motion across the retina for lower values of dot coherence. However, sensitivity was not zero, implying that visual motion should still be driving eye velocity toward target velocity. When we changed dot coherence from 100% to lower values during accurate steady-state pursuit, we observed larger eye decelerations for lower coherences, as expected if motor feedback was reduced in gain. A simple pursuit model accounts for our data based on separate modulation of the strength of visual-motor transmission and motor feedback. We suggest that reduced dot coherence allows us to observe evidence for separate modulations of the gain of visual-motor transmission during pursuit initiation and of the motor corollary discharges that comprise eye velocity memory and support steady-state tracking.NEW & NOTEWORTHY We exploit low-coherence patches of dots to control the initiation and steady state of smooth pursuit eye movements and show that these two phases of movement are modulated separately by the reliability of visual motion signals. We conclude that the neural circuit for pursuit includes separate modulation of the strength of visual-motor transmission for movement initiation and of eye velocity positive feedback to support steady-state tracking.

A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex.

SummaryTo understand how the brain processes sensory information to guide behavior, we must know how stimulus representations are transformed throughout the visual cortex. Here we report an open, large-scale physiological survey of activity in the awake mouse visual cortex: the Allen Brain Observatory Visual Coding dataset. This publicly available dataset includes cortical activity from nearly 60,000 neurons from 6 visual areas, 4 layers, and 12 transgenic mouse lines from 243 adult mice, in response to a systematic set of visual stimuli. We classify neurons based on joint reliabilities to multiple stimuli and validate this functional classification with models of visual responses. While most classes are characterized by responses to specific subsets of the stimuli, the largest class is not reliably responsive to any of the stimuli and becomes progressively larger in higher visual areas. These classes reveal a functional organization wherein putative dorsal areas show specialization for visual motion signals.

The Neural Basis for Response Latency in a Sensory-Motor Behavior.

The motor cortex for smooth pursuit eye movements contributes to sensory-motor reaction time through the amplitude of preparatory activity and the latency of transient, visually driven responses.

Pigeons integrate visual motion signals differently than humans

Perceiving motion is a fundamental ability for animals. Primates integrate local 1D motion across orientation and space to compute a rigid 2D motion. It is unknown whether the rule of 2D motion integration is universal within the vertebrate clade; comparative studies of animals with different ecological backgrounds from primates may help answer that question. Here we investigated 2D motion integration in pigeons, using hierarchically structured motion stimuli, namely a barber-pole illusion and plaid motion. The pigeons were trained to report the direction of motion of random dots. When a barber-pole or plaid stimulus was presented, they reported the direction perpendicular to the grating orientation for barber-pole and the vector average of two component gratings for plaid motion. These results demonstrate that pigeons perceive different directions than humans from the same motion stimuli, and suggest that the 2D integrating rules in the primate brain has been elaborated through phylogenetic or ecological factors specific to the clade.

Fake hand in movement: Visual motion cues from the rubber hand are processed for kinesthesia.

The feeling that a fake (e.g. rubber) hand belongs to a person's own body can be elicited by synchronously stroking the fake hand and the real hand, with the latter hidden from view. Here, we sought to determine whether visual motion signals from that incorporated rubber hand would provide relevant cues for sensing movement (i.e. kinesthesia). After 180 s of visuo-tactile synchronous or asynchronous stroking, the fake hand was moved along the lateral or the sagittal axis. After synchronous stroking, movement of the rubber hand induced illusory movement of the static (real) hand in the same direction; the illusion was slightly more frequent and more intense when the fake hand was moved along the sagittal axis. We therefore conclude that visual signals of motion originating from the rubber hand are integrated for kinesthesia by the central nervous system just as visual signals from the real hand are.

Tactile motion biases visual motion perception in binocular rivalry.

There is an ongoing debate whether or not multisensory interactions require awareness of the sensory signals. Static visual and tactile stimuli have been shown to influence each other even in the absence of visual awareness. However, it is unclear if this finding generalizes to dynamic contexts. In the present study, we presented visual and tactile motion stimuli and induced fluctuations of visual awareness by means of binocular rivalry: two gratings which drifted in opposite directions were displayed, one to each eye. One visual motion stimulus dominated and reached awareness while the other visual stimulus was suppressed from awareness. Tactile motion stimuli were presented at random time points during the visual stimulation. The motion direction of a tactile stimulus alwaysmatched the direction of one of the concurrently presented visual stimuli. The visual gratings were differently tinted, and participants reported the color of the currently seen stimulus. Tactile motion delayed perceptual switches that ended dominance periods of congruently moving visual stimuli compared to switches duringvisual-only stimulation. In addition, tactile motion fostered the return to dominance of suppressed, congruently moving visual stimuli, but only if the tactile motion started at a late stage of the ongoing visual suppression period. At later stages, perceptual suppression is typically decreasing. These results suggest that visual awareness facilitates but does not gate multisensory interactions between visual and tactile motion signals.

Eye Position Error Influence over "Open-Loop" Smooth Pursuit Initiation.

The oculomotor system integrates a variety of visual signals into appropriate motor plans, but such integration can have widely varying time scales. For example, smooth pursuit eye movements to follow a moving target are slower and longer lasting than saccadic eye movements and it has been suggested that initiating a smooth pursuit eye movement involves an obligatory "open-loop" interval in which new visual motion signals presumably cannot influence the ensuing motor plan for up to 100 ms after movement initiation. However, this view is contrary to the idea that the oculomotor periphery has privileged access to short-latency visual signals. Here, we show that smooth pursuit initiation is sensitive to visual inputs, even in open-loop intervals. We instructed male rhesus macaque monkeys to initiate saccade-free smooth pursuit eye movements and injected a transient, instantaneous eye position error signal at different times relative to movement initiation. We found robust short-latency modulations in eye velocity and acceleration, starting only ∼50 ms after transient signal occurrence and even during open-loop pursuit initiation. Critically, the spatial direction of the injected position error signal had predictable effects on smooth pursuit initiation, with forward errors increasing eye acceleration and backward errors reducing it. Catch-up saccade frequencies and amplitudes were also similarly altered ∼50 ms after transient signals, much like the well known effects on microsaccades during fixation. Our results demonstrate that smooth pursuit initiation is highly sensitive to visual signals and that catch-up saccade generation is reset after a visual transient.SIGNIFICANCE STATEMENT Smooth pursuit eye movements allow us to track moving objects. The first ∼100 ms of smooth pursuit initiation are characterized by smooth eye acceleration and are overwhelmingly described as being "open-loop"; that is, unmodifiable by new visual motion signals. We found that all phases of smooth pursuit, including the so-called open-loop intervals, are reliably modifiable by visual signals. We injected transient flashes resulting in very brief, spatially specific position error signals to smooth pursuit and observed very short-latency changes in smooth eye movements to minimize such errors. Our results highlight the flexibility of the oculomotor system in reacting to environmental events and suggest a functional role for the pervasiveness of visual sensitivity in oculomotor control brain regions.

Encoding of Reward and Decoding Movement from the Frontal Eye Field during Smooth Pursuit Eye Movements.

Expectation of reward potentiates sensorimotor transformations to drive vigorous movements. One of the main challenges in studying reward is to determine how representations of reward interact with the computations that drive behavior. We recorded activity in smooth pursuit neurons in the frontal eye field (FEF) of two male rhesus monkeys while controlling the eye speed by manipulating either reward size or target speed. The neurons encoded the different reward conditions more strongly than the different target speed conditions. This pattern could not be explained by differences in the eye speed, since the eye speed sensitivity of the neurons was also larger for the reward conditions. Pooling the responses by the preferred direction of the neurons attenuated the reward modulation and led to a tighter association between neural activity and behavior. Therefore, a plausible decoder such as the population vector could explain how the FEF both drives behavior and encodes reward beyond behavior.SIGNIFICANCE STATEMENT Motor areas combine sensory and reward information to drive movement. To disambiguate these sources, we manipulated the speed of smooth pursuit eye movements by controlling either the size of the reward or the speed of the visual motion signals. We found that the relationship between activity in frontal eye field and eye kinematics varied: the eye speed sensitivity was larger for the different reward conditions than for the different target speed conditions. Decoders that pooled signals by the preferred direction of the neurons attenuated the reward modulations. These decoders may indicate how reward can be both encoded beyond eye kinematics at the single neuron level and drive movement at the population level.