Natural Optic Flow Research Articles

Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation

Animals benefit from knowing if and how they are moving. Across the animal kingdom, sensory information in the form of optic flow over the visual field is used to estimate self-motion. However, different species exhibit strong spatial biases in how they use optic flow. Here, we show computationally that noisy natural environments favor visual systems that extract spatially biased samples of optic flow when estimating self-motion. The performance associated with these biases, however, depends on interactions between the environment and the animal's brain and behavior. Using the larval zebrafish as a model, we recorded natural optic flow associated with swimming trajectories in the animal's habitat with an omnidirectional camera mounted on a mechanical arm. An analysis of these flow fields suggests that lateral regions of the lower visual field are most informative about swimming speed. This pattern is consistent with the recent findings that zebrafish optomotor responses are preferentially driven by optic flow in the lateral lower visual field, which we extend with behavioral results from a high-resolution spherical arena. Spatial biases in optic-flow sampling are likely pervasive because they are an effective strategy for determining self-motion in noisy natural environments.

Sparse coding predicts optic flow specificities of zebrafish pretectal neurons

Zebrafish pretectal neurons exhibit specificities for large-field optic flow patterns associated with rotatory or translatory body motion. We investigate the hypothesis that these specificities reflect the input statistics of natural optic flow. Realistic motion sequences were generated using computer graphics simulating self-motion in an underwater scene. Local retinal motion was estimated with a motion detector and encoded in four populations of directionally tuned retinal ganglion cells, represented as two signed input variables. This activity was then used as input into one of three learning networks: a sparse coding network (competitive learning), PCA whitening with subsequent sparse coding, and a backpropagation network (supervised learning). All simulations developed specificities for optic flow which are comparable to those found in a neurophysiological study (Kubo et al. in Neuron 81(6):1344–1359, 2016. https://doi.org/10.1016/j.neuron.2014.02.043), but relative frequencies of the various neuronal responses were best modeled by the sparse coding approach without whitening. We conclude that the optic flow neurons in the zebrafish pretectum do reflect the optic flow statistics. The predicted vectorial receptive fields show not only typical optic flow fields but also “Gabor” and dipole-shaped patterns that likely reflect difference fields needed for reconstruction by linear superposition.

Optic flow instructs retinotopic map formation through a spatial to temporal to spatial transformation of visual information.

Retinotopic maps are plastic in response to changes in sensory input; however, the experience-dependent instructive cues that organize retinotopy are unclear. In animals with forward-directed locomotion, the predominant anterior to posterior optic flow activates retinal ganglion cells in a stereotyped temporal to nasal sequence. Here we imaged retinotectal axon arbor location and structural plasticity to assess map refinement in vivo while exposing Xenopus tadpoles to visual stimuli. We show that the temporal sequence of retinal activity driven by natural optic flow organizes retinotopy by regulating axon arbor branch dynamics, whereas the opposite sequence of retinal activity prevents map refinement. Our study demonstrates that a spatial to temporal to spatial transformation of visual information controls experience-dependent topographic map plasticity. This organizational principle is likely to apply to other sensory modalities and projections in the brain.

Diverse Suppressive Influences in Area MT and Selectivity to Complex Motion Features

Neuronal selectivity results from both excitatory and suppressive inputs to a given neuron. Suppressive influences can often significantly modulate neuronal responses and impart novel selectivity in the context of behaviorally relevant stimuli. In this work, we use a naturalistic optic flow stimulus to explore the responses of neurons in the middle temporal area (MT) of the alert macaque monkey; these responses are interpreted using a hierarchical model that incorporates relevant nonlinear properties of upstream processing in the primary visual cortex (V1). In this stimulus context, MT neuron responses can be predicted from distinct excitatory and suppressive components. Excitation is spatially localized and matches the measured preferred direction of each neuron. Suppression is typically composed of two distinct components: (1) a directionally untuned component, which appears to play the role of surround suppression and normalization; and (2) a direction-selective component, with comparable tuning width as excitation and a distinct spatial footprint that is usually partially overlapping with excitation. The direction preference of this direction-tuned suppression varies widely across MT neurons: approximately one-third have overlapping suppression in the opposite direction as excitation, and many other neurons have suppression with similar direction preferences to excitation. There is also a population of MT neurons with orthogonally oriented suppression. We demonstrate that direction-selective suppression can impart selectivity of MT neurons to more complex velocity fields and that it can be used for improved estimation of the three-dimensional velocity of moving objects. Thus, considering MT neurons in a complex stimulus context reveals a diverse set of computations likely relevant for visual processing in natural visual contexts.

Encoding of naturalistic optic flow by motion sensitive neurons of nucleus rotundus in the zebra finch (Taeniopygia guttata)

The retinal image changes that occur during locomotion, the optic flow, carry information about self-motion and the three-dimensional structure of the environment. Especially fast moving animals with only little binocular vision depend on these depth cues for maneuvering. They actively control their gaze to facilitate perception of depth based on cues in the optic flow. In the visual system of birds, nucleus rotundus neurons were originally found to respond to object motion but not to background motion. However, when background and object were both moving, responses increased the more the direction and velocity of object and background motion on the retina differed. These properties may play a role in representing depth cues in the optic flow. We therefore investigated, how neurons in nucleus rotundus respond to optic flow that contains depth cues. We presented simplified and naturalistic optic flow on a panoramic LED display while recording from single neurons in nucleus rotundus of anaesthetized zebra finches. Unlike most studies on motion vision in birds, our stimuli included depth information. We found extensive responses of motion selective neurons in nucleus rotundus to optic flow stimuli. Simplified stimuli revealed preferences for optic flow reflecting translational or rotational self-motion. Naturalistic optic flow stimuli elicited complex response modulations, but the presence of objects was signaled by only few neurons. The neurons that did respond to objects in the optic flow, however, show interesting properties.

Octopaminergic modulation of a fly visual motion-sensitive neuron during stimulation with naturalistic optic flow

In a variety of species locomotor activity, like walking or flying, has been demonstrated to alter visual information processing. The neuromodulator octopamine was shown to change the response characteristics of optic flow processing neurons in the fly's visual system in a similar way as locomotor activity. This modulation resulted in enhanced neuronal responses, in particular during sustained stimulation with high temporal frequencies, and in shorter latencies of responses to abrupt onsets of pattern motion. These state-dependent changes were interpreted to adjust neuronal tuning to the range of high velocities encountered during locomotion. Here we assess the significance of these changes for the processing of optic flow as experienced during flight. Naturalistic image sequences were reconstructed based on measurements of the head position and gaze direction of Calliphora vicina flying in an arena. We recorded the responses of the V1 neuron during presentation of these image sequences on a panoramic stimulus device (“FliMax”). Consistent with previous accounts, we found that spontaneous as well as stimulus-induced spike rates were increased by an octopamine agonist and decreased by an antagonist. Moreover, a small but consistent decrease in response latency upon octopaminergic activation was present, which might support fast responses to optic flow cues and limit instabilities during closed-loop optomotor regulation. However, apart from these effects the similarities between the dynamic response properties in the different pharmacologically induced states were surprisingly high, indicating that the processing of naturalistic optic flow is not fundamentally altered by octopaminergic modulation.

Octopaminergic modulation of neuronal responses to flight-induced naturalistic image sequences

Event Abstract Back to Event Octopaminergic modulation of neuronal responses to flight-induced naturalistic image sequences Diana Rien1, Roland Kern1 and Rafael Kurtz1* 1 Bielefeld University, Neurobiology, Germany Sensory input experienced by animals during behavioral action may differ fundamentally during different states of locomotor activity. Given the limited working range of neurons, adjustments of their sensitivity and stimulus tuning to the current locomotor state might be advantageous. Such state-dependent modulation, which has recently received much attention in various animal models (1), was demonstrated in visual motion-sensitive neurons of different fly species (2-4). The modulation of these neurons is mediated by octopamine, as was recently shown directly by manipulating the activity of a set of octopaminergic neurons (5). Flight or walking activity, as well as application of an octopamine agonist caused enhanced resting activity, shorter latencies in the response to abrupt motion onsets, and larger stimulus-induced responses (2,3,5-10). The strongest response boosts were observed with sustained presentation of high temporal frequencies, thus leading in some studies to a shift in the temporal frequency tuning toward higher values (2,7). However, the functional relevance of these state-dependent changes is unclear, because experimenter-defined stimuli have been used instead of natural optic flow. We assessed the functional significance of octopaminergic neuromodulation for the processing of optic flow as experienced during flight. Naturalistic image sequences were reconstructed based on the head position and gaze orientation during flight (11) and replayed with a panoramic stimulus device (12). We studied how the responses of the V1 neuron in Calliphora vicina to these stimuli are modified by octopaminergic modulation. Spontaneous as well as stimulus- induced spike rates were increased by the octopamine agonist CDM and decreased by an antagonist, epinastine. Moreover, small but consistent shifts in the peak positions of cross- correlograms between the different drug-induced states were present, consistent with an octopamine-induced acceleration of responses to optic flow cues. However, cross-correlations as well as coherence values between the responses recorded in different states of octopaminergic modulation were almost as high as those between traces obtained within one state. Thus, our results suggest that naturalistic optic-flow processing is slightly modified, but not fundamentally altered, by state-dependent neuromodulation. Figure 1

Object representation and distance encoding in three-dimensional environments by a neural circuit in the visual system of the blowfly

Three motion-sensitive key elements of a neural circuit, presumably involved in processing object and distance information, were analyzed with optic flow sequences as experienced by blowflies in a three-dimensional environment. This optic flow is largely shaped by the blowfly's saccadic flight and gaze strategy, which separates translational flight segments from fast saccadic rotations. By modifying this naturalistic optic flow, all three analyzed neurons could be shown to respond during the intersaccadic intervals not only to nearby objects but also to changes in the distance to background structures. In the presence of strong background motion, the three types of neuron differ in their sensitivity for object motion. Object-induced response increments are largest in FD1, a neuron long known to respond better to moving objects than to spatially extended motion patterns, but weakest in VCH, a neuron that integrates wide-field motion from both eyes and, by inhibiting the FD1 cell, is responsible for its object preference. Small but significant object-induced response increments are present in HS cells, which serve both as a major input neuron of VCH and as output neurons of the visual system. In both HS and FD1, intersaccadic background responses decrease with increasing distance to the animal, although much more prominently in FD1. This strong dependence of FD1 on background distance is concluded to be the consequence of the activity of VCH that dramatically increases its activity and, thus, its inhibitory strength with increasing distance.

Neuronal encoding of object and distance information: a model simulation study on naturalistic optic flow processing.

We developed a model of the input circuitry of the FD1 cell, an identified motion-sensitive interneuron in the blowfly's visual system. The model circuit successfully reproduces the FD1 cell's most conspicuous property: its larger responses to objects than to spatially extended patterns. The model circuit also mimics the time-dependent responses of FD1 to dynamically complex naturalistic stimuli, shaped by the blowfly's saccadic flight and gaze strategy: the FD1 responses are enhanced when, as a consequence of self-motion, a nearby object crosses the receptive field during intersaccadic intervals. Moreover, the model predicts that these object-induced responses are superimposed by pronounced pattern-dependent fluctuations during movements on virtual test flights in a three-dimensional environment with systematic modifications of the environmental patterns. Hence, the FD1 cell is predicted to detect not unambiguously objects defined by the spatial layout of the environment, but to be also sensitive to objects distinguished by textural features. These ambiguous detection abilities suggest an encoding of information about objects—irrespective of the features by which the objects are defined—by a population of cells, with the FD1 cell presumably playing a prominent role in such an ensemble.

Species-Specific Flight Styles of Flies are Reflected in the Response Dynamics of a Homolog Motion-Sensitive Neuron.

Hoverflies and blowflies have distinctly different flight styles. Yet, both species have been shown to structure their flight behavior in a way that facilitates extraction of 3D information from the image flow on the retina (optic flow). Neuronal candidates to analyze the optic flow are the tangential cells in the third optical ganglion – the lobula complex. These neurons are directionally selective and integrate the optic flow over large parts of the visual field. Homolog tangential cells in hoverflies and blowflies have a similar morphology. Because blowflies and hoverflies have similar neuronal layout but distinctly different flight behaviors, they are an ideal substrate to pinpoint potential neuronal adaptations to the different flight styles. In this article we describe the relationship between locomotion behavior and motion vision on three different levels: (1) We compare the different flight styles based on the categorization of flight behavior into prototypical movements. (2) We measure the species-specific dynamics of the optic flow under naturalistic flight conditions. We found the translational optic flow of both species to be very different. (3) We describe possible adaptations of a homolog motion-sensitive neuron. We stimulate this cell in blowflies (Calliphora) and hoverflies (Eristalis) with naturalistic optic flow generated by both species during free flight. The characterized hoverfly tangential cell responds faster to transient changes in the optic flow than its blowfly homolog. It is discussed whether and how the different dynamical response properties aid optic flow analysis.

Manipulating the content of dynamic natural scenes to characterize response in human MT/MST

Optic flow is one of the most important sources of information for enabling human navigation through the world. A striking finding from single-cell studies in monkeys is the rapid saturation of response of MT/MST areas with the density of optic flow type motion information. These results are reflected psychophysically in human perception in the saturation of motion aftereffects. We began by comparing responses to natural optic flow scenes in human visual brain areas to responses to the same scenes with inverted contrast (photo negative). This changes scene familiarity while preserving local motion signals. This manipulation had no effect; however, the response was only correlated with the density of local motion (calculated by a motion correlation model) in V1, not in MT/MST. To further investigate this, we manipulated the visible proportion of natural dynamic scenes and found that areas MT and MST did not increase in response over a 16-fold increase in the amount of information presented, i.e., response had saturated. This makes sense in light of the sparseness of motion information in natural scenes, suggesting that the human brain is well adapted to exploit a small amount of dynamic signal and extract information important for survival.

Impact of visual motion adaptation on neural responses to objects and its dependence on the temporal characteristics of optic flow

It is still unclear how sensory systems efficiently encode signals with statistics as experienced by animals in the real world and what role adaptation plays during normal behavior. Therefore, we studied the performance of visual motion-sensitive neurons of blowflies, the horizontal system neurons, with optic flow that was reconstructed from the head trajectories of semi-free-flying flies. To test how motion adaptation is affected by optic flow dynamics, we manipulated the seminatural optic flow by targeted modifications of the flight trajectories and assessed to what extent neuronal responses to an object located close to the flight trajectory depend on adaptation dynamics. For all types of adapting optic flow object-induced response increments were stronger in the adapted compared with the nonadapted state. Adaptation with optic flow characterized by the typical alternation between translational and rotational segments produced this effect but also adaptation with optic flow that lacked these distinguishing features and even pure rotation at a constant angular velocity. The enhancement of object-induced response increments had a direction-selective component because preferred-direction rotation and natural optic flow were more efficient adaptors than null-direction rotation. These results indicate that natural dynamics of optic flow is not a basic requirement to adapt neurons in a specific, presumably functionally beneficial way. Our findings are discussed in the light of adaptation mechanisms proposed on the basis of experiments previously done with conventional experimenter-defined stimuli.

Local statistics of retinal optic flow for self-motion through natural sceneries

Image analysis in the visual system is well adapted to the statistics of natural scenes. Investigations of natural image statistics have so far mainly focused on static features. The present study is dedicated to the measurement and the analysis of the statistics of optic flow generated on the retina during locomotion through natural environments. Natural locomotion includes bouncing and swaying of the head and eye movement reflexes that stabilize gaze onto interesting objects in the scene while walking. We investigate the dependencies of the local statistics of optic flow on the depth structure of the natural environment and on the ego-motion parameters. To measure these dependencies we estimate the mutual information between correlated data sets. We analyze the results with respect to the variation of the dependencies over the visual field, since the visual motions in the optic flow vary depending on visual field position. We find that retinal flow direction and retinal speed show only minor statistical interdependencies. Retinal speed is statistically tightly connected to the depth structure of the scene. Retinal flow direction is statistically mostly driven by the relation between the direction of gaze and the direction of ego-motion. These dependencies differ at different visual field positions such that certain areas of the visual field provide more information about ego-motion and other areas provide more information about depth. The statistical properties of natural optic flow may be used to tune the performance of artificial vision systems based on human imitating behavior, and may be useful for analyzing properties of natural vision systems.

Encoding of Naturalistic Optic Flow by a Population of Blowfly Motion-Sensitive Neurons

In sensory systems information is encoded by the activity of populations of neurons. To analyze the coding properties of neuronal populations sensory stimuli have usually been used that were much simpler than those encountered in real life. It has been possible only recently to stimulate visual interneurons of the blowfly with naturalistic visual stimuli reconstructed from eye movements measured during free flight. Therefore we now investigate with naturalistic optic flow the coding properties of a small neuronal population of identified visual interneurons in the blowfly, the so-called VS and HS neurons. These neurons are motion sensitive and directionally selective and are assumed to extract information about the animal's self-motion from optic flow. We could show that neuronal responses of VS and HS neurons are mainly shaped by the characteristic dynamical properties of the fly's saccadic flight and gaze strategy. Individual neurons encode information about both the rotational and the translational components of the animal's self-motion. Thus the information carried by individual neurons is ambiguous. The ambiguities can be reduced by considering neuronal population activity. The joint responses of different subpopulations of VS and HS neurons can provide unambiguous information about the three rotational and the three translational components of the animal's self-motion and also, indirectly, about the three-dimensional layout of the environment.

Motion Adaptation Leads to Parsimonious Encoding of Natural Optic Flow by Blowfly Motion Vision System

Neurons sensitive to visual motion change their response properties during prolonged motion stimulation. These changes have been interpreted as adaptive and were concluded, for instance, to adjust the sensitivity of the visual motion pathway to velocity changes or to increase the reliability of encoding of motion information. These conclusions are based on experiments with experimenter-designed motion stimuli that differ substantially with respect to their dynamical properties from the optic flow an animal experiences during normal behavior. We analyze for the first time motion adaptation under natural stimulus conditions. The experiments are done on the H1-cell, an identified neuron in the blowfly visual motion pathway that has served in many previous studies as a model system for visual motion computation. We reconstructed optic flow perceived by a blowfly in free flight and used this behaviorally generated optic flow to study motion adaptation. A variety of measures (variability in spike count, response latency, jitter of spike timing) suggests that the coding quality does not improve with prolonged stimulation. However, although the number of spikes decreases considerably during stimulation with natural optic flow, the amount of information that is conveyed stays nearly constant. Thus the information per spike increases, and motion adaptation leads to parsimonious coding without sacrificing the reliability with which behaviorally relevant information is encoded.

Responses of blowfly motion-sensitive neurons to reconstructed optic flow along outdoor flight paths

The retinal image flow a blowfly experiences in its daily life on the wing is determined by both the structure of the environment and the animal's own movements. To understand the design of visual processing mechanisms, there is thus a need to analyse the performance of neurons under natural operating conditions. To this end, we recorded flight paths of flies outdoors and reconstructed what they had seen, by moving a panoramic camera along exactly the same paths. The reconstructed image sequences were later replayed on a fast, panoramic flight simulator to identified, motion sensitive neurons of the so-called horizontal system (HS) in the lobula plate of the blowfly, which are assumed to extract self-motion parameters from optic flow. We show that under real life conditions HS-cells not only encode information about self-rotation, but are also sensitive to translational optic flow and, thus, indirectly signal information about the depth structure of the environment. These properties do not require an elaboration of the known model of these neurons, because the natural optic flow sequences generate--at least qualitatively--the same depth-related response properties when used as input to a computational HS-cell model and to real neurons.

On the computations analyzing natural optic flow: quantitative model analysis of the blowfly motion vision pathway.

For many animals, including humans, the optic flow generated on the eyes during locomotion is an important source of information about self-motion and the structure of the environment. The blowfly has been used frequently as a model system for experimental analysis of optic flow processing at the microcircuit level. Here, we describe a model of the computational mechanisms implemented by these circuits in the blowfly motion vision pathway. Although this model was originally proposed based on simple experimenter-designed stimuli, we show that it is also capable to quantitatively predict the responses to the complex dynamic stimuli a blowfly encounters in free flight. In particular, the model visual system exploits the active saccadic gaze and flight strategy of blowflies in a similar way, as does its neuronal counterpart. The model circuit extracts information about translation velocity in the intersaccadic intervals and thus, indirectly, about the three-dimensional layout of the environment. By stepwise dissection of the model circuit, we determine which of its components are essential for these remarkable features. When accounting for the responses to complex natural stimuli, the model is much more robust against parameter changes than when explaining the neuronal responses to simple experimenter-defined stimuli. In contrast to conclusions drawn from experiments with simple stimuli, optimization of the parameter set for different segments of natural optic flow stimuli do not indicate pronounced adaptational changes of these parameters during long-lasting stimulation.

Movement-induced motion signal distributions in outdoor scenes

The movement of an observer generates a characteristic field of velocity vectors on the retina (). Because such optic flow-fields are useful for navigation, many theoretical, psychophysical and physiological studies have addressed the question how egomotion parameters such as direction of heading can be estimated from optic flow. Little is known, however, about the structure of optic flow under natural conditions. To address this issue, we recorded sequences of panoramic images along accurately defined paths in a variety of outdoor locations and used these sequences as input to a two-dimensional array of correlation-based motion detectors (2DMD). We find that (a) motion signal distributions are sparse and noisy with respect to local motion directions; (b) motion signal distributions contain patches (motion streaks) which are systematically oriented along the principal flow-field directions; (c) motion signal distributions show a distinct, dorso-ventral topography, reflecting the distance anisotropy of terrestrial environments; (d) the spatiotemporal tuning of the local motion detector we used has little influence on the structure of motion signal distributions, at least for the range of conditions we tested; and (e) environmental motion is locally noisy throughout the visual field, with little spatial or temporal correlation; it can therefore be removed by temporal averaging and is largely over-ridden by image motion caused by observer movement. Our results suggest that spatial or temporal integration is important to retrieve reliable information on the local direction and size of motion vectors, because the structure of optic flow is clearly detectable in the temporal average of motion signal distributions. Egomotion parameters can be reliably retrieved from such averaged distributions under a range of environmental conditions. These observations raise a number of questions about the role of specific environmental and computational constraints in the processing of natural optic flow.

Neural Processing of Naturalistic Optic Flow

Stimuli traditionally used for analyzing visual information processing are much simpler than what an animal sees in normal life. When characterized with traditional stimuli, neuronal responses were found to depend on various parameters such as contrast, texture, or velocity of motion, and thus were highly ambiguous. In behavioral situations, all of these parameters change simultaneously and differently in different parts of the visual field. Thus it is hardly possible to predict from traditional analyses what information is encoded by neurons in behavioral situations. Therefore, we characterized an identified neuron in the optomotor system of the blowfly with image sequences as they were seen by animals walking in a structured environment. We conclude that during walking, the response of the neuron reflects the animal's turning direction nearly independently of the texture and spatial layout of the environment. Our findings stress the significance of analyzing the performance of neuronal circuits under their natural operating conditions.

Neuronal responses in the motion pathway of the macaque monkey to natural optic flow stimuli.

Neurones in higher visual motion areas in the superior temporal sulcus (STS) of the macaque monkey respond to abstract random dot optic flow stimuli. Higher motion areas may not only represent, but in a next computational stage also analyse the flow field to determine, for instance, the direction of heading for navigation purposes. Real world visual scenes differ in several aspects from these abstract optic flow stimuli. We tested the neuronal response to naturalistic optic flow stimuli which simulated egomotion in different virtual environments and contained different numbers of visual cues. Neuronal activity depended mainly on the position of the focus of expansion rather than on other visual cues. This finding supports the hypothesis that higher motion areas within the STS analyse optic flow in natural scenes and can thus signal the direction of heading.