Optic Flow Processing Research Articles

Seeing the World as it is: Mimicking Veridical Motion Perception in Schizophrenia Using Non-invasive Brain Stimulation in Healthy Participants

Schizophrenia (Sz) is a mental health disorder characterized by severe cognitive, emotional, social, and perceptual deficits. Visual deficits are found in tasks relying on the magnocellular/dorsal stream. In our first experiment we established deficits in global motion processing in Sz patients compared to healthy controls. We used a novel task in which background optic flow produces a distortion of the apparent trajectory of a moving stimulus, leading control participants to provide biased estimates of the true motion trajectory under conditions of global stimulation. Sz patients were significantly less affected by the global background motion, and reported trajectories that were more veridically accurate than those of controls. In order to study the mechanism of this effect, we performed a second experiment where we applied transcranial electrical stimulation over area MT+ to selectively modify global motion processing of optic flow displays in healthy participants. Cathodal and high frequency random noise stimulation had opposite effects on trajectory perception in optic flow. The brain stimulation over a control site and in a control task revealed that the effect of stimulation was specific for global motion processing in area MT+. These findings both support prior studies of impaired early visual processing in Sz and provide novel approaches for measurement and manipulation of the underlying circuits.

Optomotor steering and flight control requires a specific sub-section of the compound eye in the hawkmoth, Manduca sexta.

While tracking odor plumes, male hawkmoths use optic flow cues to stabilize their flight movements with respect to their environment. We studied the responses of freely flying moths tracking odor plumes in a laboratory wind tunnel and tethered moths in an optomotor flight simulator to determine the locations on the compound eye on which critical optic flow cues are detected. In these behavioral experiments, we occluded specific regions of the compound eye and systematically examined the moths' behavior for specific deficits in optic flow processing. Freely flying moths with the dorsal half of the compound eye painted were unable to maintain stable flight and track the wind-borne odor plume. However, the plume tracking performance of moths with the ventral half of their compound eyes painted was the same as unpainted controls. In a matched set of experiments, we presented tethered moths with moving vertically oriented sinusoidal gratings and found that individuals with their eyes unpainted, ventrally painted and medially painted all responded by attempting optomotor-driven turns in the same proportion. In contrast, individuals with their compound eyes dorsally painted, laterally painted and completely painted showed no optomotor turning response. We decreased the contrast of the visual stimulus and found that this relationship was consistent down to a contrast level of 2.5%. We conclude that visual input from the dorso-lateral region of the moth's visual world is critical for successful maintenance of flight stability and that this species' visual environment must meet or exceed a contrast ratio of 2.5% to support visual flight control.

Functional Neuroanatomy of Time-To-Passage Perception

The time until an approaching object passes the observer is referred to as time-to-passage (TTP). Accurate judgment of TTP is critical for visually guided navigation, such as when walking, riding a bicycle, or driving a car. Previous research has shown that observers are able to make TTP judgments in the absence of information about local retinal object expansion. In this paper we combine psychophysics and functional MRI (fMRI) to investigate the neural substrate of TTP processing. In a previous psychophysical study, we demonstrated that when local retinal expansion cues are not available, observers take advantage of multiple sources of information to judge TTP, such as optic flow and object retinal velocities, and integrate these cues through a flexible and economic strategy. To induce strategy changes, we introduced trials with motion but without coherent optic flow (0% coherence of the background), and trials with coherent, but noisy, optic flow (75% coherence of the background). In a functional magnetic resonance imaging (fMRI) study we found that coherent optic flow cues resulted in better behavioral performance as well as higher and broader cortical activations across the visual motion processing pathway. Blood oxygen-level-dependent (BOLD) signal changes showed significant involvement of optic flow processing in the precentral sulcus (PreCS), postcentral sulcus (PostCS) and middle temporal gyrus (MTG) across all conditions. Not only highly activated during motion processing, bilateral hMT areas also showed a complex pattern in TTP judgment processing, which reflected a flexible TTP response strategy.

Spike Burst Coding of Translatory Optic Flow and Depth from Motion in the Fly Visual System

Many animals use the visual motion generated by traveling straight-the translatory optic flow-to successfully navigate obstacles: near objects appear larger and to move more quickly than distant objects. Flies are expert at navigating cluttered environments, and while their visual processing of rotatory optic flow is understood in exquisite detail, how they process translatory optic flow remains a mystery. We present novel cell types that have local motion receptive fields matched to translation self-motion, the vertical translation (VT) cells. One of these, the VT1 cell, encodes self-motion in the forward-sideslip direction and fires action potentials in spike bursts as well as single spikes. We show that the spike burst coding is size and speed-tuned and is selectively modulated by motion parallax-the relative motion experienced during translation. These properties are spatially organized, so that the cell is most excited by clutter rather than isolated objects. When the fly is presented with a simulation of flying past an elevated object, the spike burst activity is modulated by the height of the object, and the rate of single spikes is unaffected. When the moving object alone is experienced, the cell is weakly driven. Meanwhile, the VT2-3 cells have motion receptive fields matched to the lift axis. In conjunction with previously described horizontal cells, the VT cells have properties well suited to the visual navigation of clutter and to encode the fly's movements along near cardinal axes of thrust, lift, and forward sideslip.

Computational neuroanatomy of human stratum proprium of interparietal sulcus

Recent advances in diffusion-weighted MRI (dMRI) and tractography have enabled identification of major long-range white matter tracts in the human brain. Yet, our understanding of shorter tracts, such as those within the parietal lobe, remains limited. Over a century ago, a tract connecting the superior and inferior parts of the parietal cortex was identified in a post-mortem study: stratum proprium of interparietal sulcus (SIPS; Sachs, Das hemisphärenmark des menschlichen grosshirns. Verlag von georg thieme, Leipzig, 1892). The tract has since been replicated in another fibre dissection study (Vergani et al., Cortex 56:145–156, 2014), however, it has not been fully investigated in the living human brain and its precise anatomical properties are yet to be described. We used dMRI and tractography to identify and characterise SIPS in vivo, and explored its spatial proximity to the cortical areas associated with optic-flow processing using fMRI. SIPS was identified bilaterally in all subjects, and its anatomical position and trajectory are consistent with previous post-mortem studies. Subsequent evaluation of the tractography results using the linear fascicle evaluation and virtual lesion analysis yielded strong statistical evidence for SIPS. We also found that the SIPS endpoints are adjacent to the optic-flow selective areas. In sum, we show that SIPS is a short-range tract connecting the superior and inferior parts of the parietal cortex, wrapping around the intraparietal sulcus, and that it may be a crucial anatomy underlying optic-flow processing. In vivo identification and characterisation of SIPS will facilitate further research on SIPS in relation to cortical functions, their development, and diseases that affect them.

Angle of gaze and optic flow direction modulate body sway

Optic flow is a crucial signal in maintaining postural stability. We sought to investigate whether the activity of postural muscles and body sway was modulated by eye position during the view of radial optic flow stimuli. We manipulated the spatial distribution of dot speed and the fixation point position to simulate specific heading directions combined with different gaze positions. The experiments were performed using stabilometry and surface electromyography (EMG) on 24 right-handed young, healthy volunteers. Center of pressure (COP) signals were analyzed considering antero-posterior and medio-lateral oscillation, COP speed, COP area, and the prevalent direction of oscillation of body sway. We found a significant main effect of body side in all COP parameters, with the right body side showing greater oscillations. The different combinations of optic flow and eye position evoked a non-uniform direction of oscillations in females. The EMG analysis showed a significant main effect for muscle and body side. The results showed that the eye position modulated body sway without changing the activity of principal leg postural muscles, suggesting that the extraretinal input regarding the eye position is a crucial signal that needs to be integrated with perceptual optic flow processing in order to control body sway.

Gamma-Band Activities in Mouse Frontal and Visual Cortex Induced by Coherent Dot Motion

A key question within systems neuroscience is to understand how the brain encodes spatially and temporally distributed local features and binds these together into one perceptual representation. Previous works in animal and human have shown that changes in neural synchrony occur during the perceptual processing and these changes are distinguished by the emergence of gamma-band oscillations (GBO, 30–80 Hz, centered at 40 Hz). Here, we used the mouse electroencephalogram to investigate how different cortical areas play roles in perceptual processing by assessing their GBO patterns during the visual presentation of coherently/incoherently moving random-dot kinematogram and static dots display. Our results revealed that GBO in the visual cortex were strongly modulated by the moving dots regardless of the existence of a global dot coherence, whereas GBO in frontal cortex were modulated by coherence of the motion. Moreover, concurrent GBO across the multiple cortical area occur more frequently for coherently moving dots. Taken together, these findings of GBO in the mouse frontal and visual cortex are related to the perceptual binding of local features into a globally-coherent representation, suggesting the dynamic interplay across the local/distributed networks of GBO in the global processing of optic flow.

Robust size illusion produced by expanding and contracting flow fields

A new illusion is described. Randomly positioned dots moved radially within an imaginary annular window. The dots’ motion periodically changed the direction, leading to an alternating percept of expanding and contracting motion. Strikingly, the apparent size of the enclosed circular region shrank during the dots’ expanding phases and dilated during the contracting phases. We quantitatively measured the illusion, and found that the presence of energy at the local kinetic edge could not account for the illusion. Besides, we reproduced the illusion on a natural scene background seen from a first-person point of view that moved forward and backward periodically. Blurring the boundaries of motion areas could not reverse the illusion in all subjects. Taken together, our observed illusion is likely induced by optic flow processing with some components of motion contrast. Expanding or contracting dots may induce the self-motion perception of either approaching or leaving way from the circle. These will make the circle appear smaller or larger since its retinal size remains constant.

Integration of optical flow and Multi-Path-Viterbi algorithm for stereo vision

This paper proposes a novel stereo matching algorithm to solve environment sensing problems. It integrates a non-convex optical flow and Viterbi process. The non-convex optical flow employs a new adaptive weighted non-convex Total Generalized Variation (TGV) model, which can obtain sharp disparity maps. Structural similarity, total variation constraint, and a specific merging strategy are combined with the 4 bi-directional Viterbi process to improve the robustness. In the fusion of the optical flow and Viterbi process, a new occlusion processing method is incorporated in order to get more sharp disparity and more robust result. Extensive experiments are conducted to compare this algorithm with other state-of-the-art methods. Experimental results show the superiority of our algorithm.

Processing of Egomotion-Consistent Optic Flow in the Rhesus Macaque Cortex.

The cortical network that processes visual cues to self-motion was characterized with functional magnetic resonance imaging in 3 awake behaving macaques. The experimental protocol was similar to previous human studies in which the responses to a single large optic flow patch were contrasted with responses to an array of 9 similar flow patches. This distinguishes cortical regions where neurons respond to flow in their receptive fields regardless of surrounding motion from those that are sensitive to whether the overall image arises from self-motion. In all 3 animals, significant selectivity for egomotion-consistent flow was found in several areas previously associated with optic flow processing, and notably dorsal middle superior temporal area, ventral intra-parietal area, and VPS. It was also seen in areas 7a (Opt), STPm, FEFsem, FEFsac and in a region of the cingulate sulcus that may be homologous with human area CSv. Selectivity for egomotion-compatible flow was never total but was particularly strong in VPS and putative macaque CSv. Direct comparison of results with the equivalent human studies reveals several commonalities but also some differences.

Design and Implementation of 176-MHz WXGA 30-fps Real-Time Optical Flow Processor

Design and Implementation of 176-MHz WXGA 30-fps Real-Time Optical Flow Processor

Connectivity Reveals Sources of Predictive Coding Signals in Early Visual Cortex During Processing of Visual Optic Flow.

Superimposed on the visual feed-forward pathway, feedback connections convey higher level information to cortical areas lower in the hierarchy. A prominent framework for these connections is the theory of predictive coding where high-level areas send stimulus interpretations to lower level areas that compare them with sensory input. Along these lines, a growing body of neuroimaging studies shows that predictable stimuli lead to reduced blood oxygen level-dependent (BOLD) responses compared with matched nonpredictable counterparts, especially in early visual cortex (EVC) including areas V1-V3. The sources of these modulatory feedback signals are largely unknown. Here, we re-examined the robust finding of relative BOLD suppression in EVC evident during processing of coherent compared with random motion. Using functional connectivity analysis, we show an optic flow-dependent increase of functional connectivity between BOLD suppressed EVC and a network of visual motion areas including MST, V3A, V6, the cingulate sulcus visual area (CSv), and precuneus (Pc). Connectivity decreased between EVC and 2 areas known to encode heading direction: entorhinal cortex (EC) and retrosplenial cortex (RSC). Our results provide first evidence that BOLD suppression in EVC for predictable stimuli is indeed mediated by specific high-level areas, in accord with the theory of predictive coding.

Differential Responses to a Visual Self-Motion Signal in Human Medial Cortical Regions Revealed by Wide-View Stimulation.

Vision is important for estimating self-motion, which is thought to involve optic-flow processing. Here, we investigated the fMRI response profiles in visual area V6, the precuneus motion area (PcM), and the cingulate sulcus visual area (CSv)—three medial brain regions recently shown to be sensitive to optic-flow. We used wide-view stereoscopic stimulation to induce robust self-motion processing. Stimuli included static, randomly moving, and coherently moving dots (simulating forward self-motion). We varied the stimulus size and the presence of stereoscopic information. A combination of univariate and multi-voxel pattern analyses (MVPA) revealed that fMRI responses in the three regions differed from each other. The univariate analysis identified optic-flow selectivity and an effect of stimulus size in V6, PcM, and CSv, among which only CSv showed a significantly lower response to random motion stimuli compared with static conditions. Furthermore, MVPA revealed an optic-flow specific multi-voxel pattern in the PcM and CSv, where the discrimination of coherent motion from both random motion and static conditions showed above-chance prediction accuracy, but that of random motion from static conditions did not. Additionally, while area V6 successfully classified different stimulus sizes regardless of motion pattern, this classification was only partial in PcM and was absent in CSv. This may reflect the known retinotopic representation in V6 and the absence of such clear visuospatial representation in CSv. We also found significant correlations between the strength of subjective self-motion and univariate activation in all examined regions except for primary visual cortex (V1). This neuro-perceptual correlation was significantly higher for V6, PcM, and CSv when compared with V1, and higher for CSv when compared with the visual motion area hMT+. Our convergent results suggest the significant involvement of CSv in self-motion processing, which may give rise to its percept.

Activity of Caudate Nucleus Neurons in a Visual Fixation Paradigm in Behaving Cats.

Beside its motor functions, the caudate nucleus (CN), the main input structure of the basal ganglia, is also sensitive to various sensory modalities. The goal of the present study was to investigate the effects of visual stimulation on the CN by using a behaving, head-restrained, eye movement-controlled feline model developed recently for this purpose. Extracellular multielectrode recordings were made from the CN of two cats in a visual fixation paradigm applying static and dynamic stimuli. The recorded neurons were classified in three groups according to their electrophysiological properties: phasically active (PAN), tonically active (TAN) and high-firing (HFN) neurons. The response characteristics were investigated according to this classification. The PAN and TAN neurons were sensitive primarily to static stimuli, while the HFN neurons responded primarily to changes in the visual environment i.e. to optic flow and the offset of the stimuli. The HFNs were the most sensitive to visual stimulation; their responses were stronger than those of the PANs and TANs. The majority of the recorded units were insensitive to the direction of the optic flow, regardless of group, but a small number of direction-sensitive neurons were also found. Our results demonstrate that both the static and the dynamic components of the visual information are represented in the CN. Furthermore, these results provide the first piece of evidence on optic flow processing in the CN, which, in more general terms, indicates the possible role of this structure in dynamic visual information processing.

Optic-flow selective cortical sensory regions associated with self-reported states of vection.

Optic flow is one of the most important visual cues to the estimation of self-motion. It has repeatedly been demonstrated that a cortical network including visual, multisensory, and vestibular areas is implicated in processing optic flow; namely, visual areas middle temporal cortex (MT+), V6; multisensory areas ventral intra-parietal area (VIP), cingulate sulcus visual area, precuneus motion area (PcM); and vestibular areas parieto-insular vestibular cortex (PIVC) and putative area 2v (p2v). However, few studies have investigated the roles of and interaction between the optic-flow selective sensory areas within the context of self-motion perception. When visual information (i.e., optic flow) is the sole cue to computing self-motion parameters, the discrepancy amongst the sensory signals may induce an illusion of self-motion referred to as ‘vection.’ This study aimed to identify optic-flow selective sensory areas that are involved in the processing of visual cues to self-motion, by introducing vection as an index and assessing activation in which of those areas reflect vection, using functional magnetic resonance imaging. The results showed that activity in visual areas MT+ and V6, multisensory area VIP and vestibular area PIVC was significantly greater while participants were experiencing vection, as compared to when they were experiencing no vection, which may indicate that activation in MT+, V6, VIP, and PIVC reflects vection. The results also place VIP in a good position to integrate visual cues related to self-motion and vestibular information.

Contextual modulation and stimulus selectivity in extrastriate cortex.

Contextual modulation is observed throughout the visual system, using techniques ranging from single-neuron recordings to behavioral experiments. Its role in generating feature selectivity within the retina and primary visual cortex has been extensively described in the literature. Here, we describe how similar computations can also elaborate feature selectivity in the extrastriate areas of both the dorsal and ventral streams of the primate visual system. We discuss recent work that makes use of normalization models to test specific roles for contextual modulation in visual cortex function. We suggest that contextual modulation renders neuronal populations more selective for naturalistic stimuli. Specifically, we discuss contextual modulation's role in processing optic flow in areas MT and MST and for representing naturally occurring curvature and contours in areas V4 and IT. We also describe how the circuitry that supports contextual modulation is robust to variations in overall input levels. Finally, we describe how this theory relates to other hypothesized roles for contextual modulation.

The coupling of vision with locomotion in cortical blindness

Maintaining or modifying the speed and direction of locomotion requires the coupling of the locomotion with the retinal optic flow that it generates. It is shown that this essential behavioral capability, which requires on-line neural control, is preserved in the cortically blind hemifield of a hemianope. In experiments, optic flow stimuli were presented to either the normal or blind hemifield while the patient was walking on a treadmill. Little difference was found between the hemifields with respect to the coupling (i.e. co-dependency) of optic flow detection with locomotion. Even in the cortically blind hemifield, faster walking resulted in the perceptual slowing of detected optic flow, and self-selected locomotion speeds demonstrated behavioral discrimination between different optic flow speeds. The results indicate that the processing of optic flow, and thereby on-line visuo-locomotor coupling, can take place along neural pathways that function without processing in Area V1, and thus in the absence of conscious intervention. These and earlier findings suggest that optic flow and object motion are processed in parallel along with correlated non-visual locomotion signals. Extrastriate interactions may be responsible for discounting the optical effects of locomotion on the perceived direction of object motion, and maintaining visually guided self-motion.

Subcellular mapping of dendritic activity in optic flow processing neurons

Dendritic integration is a fundamental element of neuronal information processing. So far, few studies have provided a detailed spatial picture of this process, describing the properties of local dendritic activity and its subcellular organization. Here, we used 2-photon calcium imaging in optic flow processing neurons of the fly Calliphora vicina to determine the preferred location and direction of local motion cues for small branchlets throughout the entire dendrite. We found a pronounced retinotopic mapping on both the subcellular and the cell population level. In addition, dendritic branchlets residing in different layers of the neuropil were tuned to distinct directions of motion. Summing the local receptive fields of all dendritic branchlets reproduced the characteristic properties of these neurons' axonal output receptive fields. Our results corroborate the notion that the dendritic morphology of vertical system cells allows them to selectively collect local motion inputs with particular directional preferences from a spatially organized input repertoire, thus forming filters that match global patterns of optic flow. Furthermore, we suggest that the facet arrangement across the fly's eye shapes the subcellular direction tuning to local motion stimuli. These data illustrate a highly structured circuit organization as an efficient way to hard-wire a complex sensory task.

Learning to navigate in a three-dimensional world: From bees to primates

We discuss the idea that environmental factors influence the neural mechanisms that evolved to enable navigation, and propose that a capacity to learn different spatial relationship rules through experience may contribute to bicoded processing. Recent experiments show that free-flying bees can learn abstract spatial relationships, and we propose that this could be combined with optic flow processing to enable three-dimensional navigation.

Heterogeneity of calretinin expression in the avian cerebellar cortex of pigeons and relationship with zebrin II

The cerebellar cortex has a fundamental parasagittal organization that is reflected in the physiological responses of Purkinje cells, projections of Purkinje cells, afferent inputs from climbing and mossy fibres and the expression of several molecular markers. The most thoroughly studied of these molecular markers is zebrin II (ZII; a.k.a. aldolase C). ZII is differentially expressed in Purkinje cells, resulting in a pattern of sagittal stripes of high expression (ZII+ve) interdigitated with stripes of little or no expression (ZII−ve). The calcium binding protein calretinin (CR) is expressed heavily in mossy fibres terminating throughout the cerebellar cortex, but whether CR is heterogeneously expressed in parasagittal stripes, like ZII, is unknown. In this study, we examined CR expression in the cerebellum of pigeons and compared it to that of ZII. CR was expressed heavily in the granule layer in mossy fibres and their terminal rosettes. Moreover, CR is expressed heterogeneously in the granule layer such that there are sagittal stripes of heavy CR labelling (CR+ve) alternating with stripes of weaker labelling (CR−ve). The CR heterogeneity is most notable in folium IXcd and folia II–IV in the anterior lobe. In the anterior lobe, the central CR+ve stripe spanning the midline is aligned with the central ZII+ve stripe, whereas the other CR+ve stripes are aligned with the ZII−ve stripes. In IXcd, the CR+ve stripes are aligned with the ZII+ve stripes. This combination of aligned and unaligned CR+ve stripes, relative to ZII+ve stripes, differs from that of parvalbumin and other neurochemical markers, but the functional consequences of this is unclear. With respect to the posterior lobe, we suggest that the CR+ve mossy fibres to IXcd originate in two retinal recipient nuclei that are involved in the processing of optic flow.