Optic Flow Patterns Research Articles

Central tendency and serial dependence in vestibular path integration.

Path integration, the process of updating one's position using successive self-motion signals, has previously been studied using visual distance reproduction tasks in which optic flow patterns provide information about traveled distance. These studies have reported that reproduced distances show two types of systematic biases: central tendency and serial dependence. In the present study, we investigated whether these biases are also present in vestibular path integration. Participants were seated on a linear motion platform and performed a distance reproduction task in total darkness. The platform first passively moved the participant a pre-defined stimulus distance which they then actively reproduced by steering the platform back the same distance. Stimulus distances were sampled from short- and long-distance probability distributions and presented in either a randomized order or in separate blocks to study the effect of presentation context. Similar to the effects observed in visual path integration, we found that reproduced distances showed an overall positive central tendency effect as well as a positive, attractive serial dependence effect. Furthermore, reproduction behavior was affected by presentation context. These results were mostly consistent with predictions of a Bayesian Kalman-filter model, originally proposed for visual path integration.

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Efficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. To investigate how this is achieved in the visual system, predictive coding with sensorimotor mismatch detection is an attractive starting point. Indeed, experimental evidence for sensorimotor mismatch signals in early visual areas exists, but it is not understood how they are integrated into cortical networks that perform input segmentation and categorization. Our model advances a biologically plausible solution by extending predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron circuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a neural network with two generative streams. The motor-to-visual stream consists of parallel microcircuits between motor and visual areas and learns to spatially predict optic flow resulting from self-motion. The second stream bidirectionally connects a motion-selective higher visual area (mHVA) to V1, assigning a crucial role to the abundant feedback connections to V1: the maintenance of a generative model of externally caused optic flow. In the model, area mHVA learns to segment moving objects from the background, and facilitates object categorization. Based on shared neurocomputational principles across species, the model also maps onto primate visual cortex. Our work extends Hebbian predictive coding to sensorimotor settings, in which the agent actively moves - and learns to predict the consequences of its own movements.

Top-down biological motion perception does not differ between adults scoring high versus low on autism traits

The perception of biological motion is an important social cognitive ability. Models of biological motion perception recognize two processes that contribute to the perception of biological motion: a bottom-up process that binds optic-flow patterns into a coherent percept of biological motion and a top-down process that binds sequences of body-posture ‘snapshots’ over time into a fluent percept of biological motion. The vast majority of studies on autism and biological motion perception have used point-light figure stimuli, which elicit biological motion perception predominantly via bottom-up processes. Here, we investigated whether autism is associated with deviances in the top-down processing of biological motion. For this, we tested a sample of adults scoring low vs high on autism traits on a recently validated EEG paradigm in which apparent biological motion is combined with frequency tagging (Cracco et al., 2022) to dissociate between two percepts: 1) the representation of individual body postures, and 2) their temporal integration into movements. In contrast to our hypothesis, we found no evidence for a diminished temporal body posture integration in the high-scoring group. We did, however, find a group difference that suggests that adults scoring high on autism traits have a visual processing style that focuses more on a single percept (i.e. either body postures or movements, contingent on saliency) compared to adults scoring low on autism traits who instead seemed to represent the two percepts included in the paradigm in a more balanced manner. Although unexpected, this finding aligns well with the autism literature on perceptual stability.

Understanding viewpoint changes in peripheral prisms for field expansion by virtual reality simulation.

Prism field expansion is a common treatment for patients with peripheral field loss, shifting images from the blind field into the seeing field. The shifted image originates from a new viewpoint translated and rotated from the original viewpoint by the prism. To understand such viewpoint changes, we simulated two field expansion methods in virtual reality: 1) angular (i.e., rotational) field expansion and 2) linear field expansion via image crop-and-shift. Changes to object locations, sizes, and optic flow patterns by those methods were demonstrated and analyzed in both static and dynamic conditions, which may affect navigation with such field expansion devices.

"Extended Descriptive Risk-Averse Bayesian Model" a More Comprehensive Approach in Simulating Complex Biological Motion Perception.

The ability to perceive biological motion is crucial for human survival, social interactions, and communication. Over the years, researchers have studied the mechanisms and neurobiological substrates that enable this ability. In a previous study, we proposed a descriptive Bayesian simulation model to represent the dorsal pathway of the visual system, which processes motion information. The model was inspired by recent studies that questioned the impact of dynamic form cues in biological motion perception and was trained to distinguish the direction of a soccer ball from a set of complex biological motion soccer-kick stimuli. However, the model was unable to simulate the reaction times of the athletes in a credible manner, and a few subjects could not be simulated. In this current work, we implemented a novel disremembering strategy to incorporate neural adaptation at the decision-making level, which improved the model's ability to simulate the athletes' reaction times. We also introduced receptive fields to detect rotational optic flow patterns not considered in the previous model to simulate a new subject and improve the correlation between the simulation and experimental data. The findings suggest that rotational optic flow plays a critical role in the decision-making process and sheds light on how different individuals perform at different levels. The correlation analysis of human versus simulation data shows a significant, almost perfect correlation between experimental and simulated angular thresholds and slopes, respectively. The analysis also reveals a strong relation between the average reaction times of the athletes and the simulations.

Target motion misjudgments reflect a misperception of the background; revealed using continuous psychophysics.

Determining the velocities of target objects as we navigate complex environments is made more difficult by the fact that our own motion adds systematic motion signals to the visual scene. The flow-parsing hypothesis asserts that the background motion is subtracted from visual scenes in such cases as a way for the visual system to determine target motions relative to the scene. Here, we address the question of why backgrounds are only partially subtracted in lab settings. At the same time, we probe a much-neglected aspect of scene perception in flow-parsing studies, that is, the perception of the background itself. Here, we present results from three experienced psychophysical participants and one inexperienced participant who took part in three continuous psychophysics experiments. We show that, when the background optic flow pattern is composed of local elements whose motions are congruent with the global optic flow pattern, the incompleteness of the background subtraction can be entirely accounted for by a misperception of the background. When the local velocities comprising the background are randomly dispersed around the average global velocity, an additional factor is needed to explain the subtraction incompleteness. We show that a model where background perception is a result of the brain attempting to infer scene motion due to self-motion can account for these results.

71‐1: Data‐Driven and Optics‐Inspired Decomposition of Global Pupil Swim in VR/AR for an Improved Perception Model of Motion Discomfort

Users can observe dynamic distortion (or global pupil swim) during the usage of VR/AR headsets. Our earlier study correlated pupil swim to selected optic flow patterns corresponding to motion and mathematically modeled discomfort. This study decomposes global pupil swim as a linear sum of orthogonal basis patterns based on Zernike polynomials for improved applicability and for an improved perception model.

Characterization of spatiotemporal dynamics in EEG data during picture naming with optical flow patterns.

In this study, we investigate the spatiotemporal dynamics of the neural oscillations by analyzing the electric potential that arises from neural activity. We identify two types of dynamics based on the frequency and phase of oscillations: standing waves or as out-of-phase and modulated waves, which represent a combination of standing and moving waves. To characterize these dynamics, we use optical flow patterns such as sources, sinks, spirals and saddles. We compare analytical and numerical solutions with real EEG data acquired during a picture-naming task. Analytical approximation of standing waves helps us to establish some properties of pattern location and number. Specifically, sources and sinks are mainly located in the same location, while saddles are positioned between them. The number of saddles correlates with the sum of all the other patterns. These properties are confirmed in both the simulated and real EEG data. In particular, source and sink clusters in the EEG data overlap with each other with median percentages around 60%, and hence have high spatial correlation, while source/sink clusters overlap with saddle clusters in less than 1%, and have different locations. Our statistical analysis showed that saddles account for about 45% of all patterns, while the remaining patterns are present in similar proportions.

Eye movements during optic flow perception

Optic flow is an important visual cue for human perception and locomotion and naturally triggers eye movements. Here we investigate whether the perception of optic flow direction is limited or enhanced by eye movements. In Exp. 1, 23 human observers localized the focus of expansion (FOE) of an optic flow pattern; in Exp. 2, 18 observers had to detect brief visual changes at the FOE. Both tasks were completed during free viewing and fixation conditions while eye movements were recorded. Task difficulty was varied by manipulating the coherence of radial motion from the FOE (4 %-90 %). During free viewing, observers tracked the optic flow pattern with a combination of saccades and smooth eye movements. During fixation, observers nevertheless made small-scale eye movements. Despite differences in spatial scale, eye movements during free viewing and fixation were similarly directed toward the FOE (saccades) and away from the FOE (smooth tracking). Whereas FOE localization sensitivity was not affected by eye movement instructions (Exp. 1), observers’ sensitivity to detect brief changes at the FOE was 27 % higher (p <.001) during free-viewing compared to fixation (Exp. 2). This performance benefit was linked to reduced saccade endpoint errors, indicating the direct beneficial impact of foveating eye movements on performance in a fine-grain perceptual task, but not during coarse perceptual localization.

The Role of Optic Flow and Gaze Direction on Postural Control

Objective: The observers use the optic flow to control self-motion. However, the current state of knowledge indicates that it is difficult to understand how optic flow is used by the visual system without a direct measurement of the changes in the flow patterns caused by eye movements during natural behaviour. The purpose of this literature review is to highlight the importance of the integration between optic flow and eye movements for postural control. Methods: A literature review of the electronic papers through July 2022 was independently performed by three investigators. The selection of the studies was made by a search on PubMed, Scopus, and Google Scholar with two groups of selected keywords. We excluded papers performed on subjects with pathologies, children, and the elderly. Results: The results of this literature analysis highlight that eye movements are required to drive visual motion processing and heading perception in both static and dynamic contexts. Conclusion: Although we now know many neural mechanisms that process heading direction from the optic flow field, a consideration of optic flow patterns relative to gaze direction provides more detailed information on how the retinal flow field is used to control body balance. Doi: 10.28991/ESJ-2022-06-06-020 Full Text: PDF

Optic Flow Processing in Patients With Macular Degeneration.

Optic flow processing was characterized in patients with macular degeneration (MD). Twelve patients with dense bilateral scotomas and 12 age- and gender-matched control participants performed psychophysical experiments. Stimuli were dynamic random-dot kinematograms projected on a large screen. For each component of optic flow (translational, radial, and rotational), we estimated motion coherence discrimination thresholds in our participants using an adaptive Bayesian procedure. Thresholds for translational, rotational, and radial patterns were comparable between patients and their matched control participants. A negative correlation was observed in patients between the time since MD diagnosis and coherence thresholds for translational patterns. Our results suggest that in patients with MD, selectivity to optic flow patterns is preserved.

Causal contribution of optic flow signal in Macaque extrastriate visual cortex for roll perception

Optic flow is a powerful cue for inferring self-motion status which is critical for postural control, spatial orientation, locomotion and navigation. In primates, neurons in extrastriate visual cortex (MSTd) are predominantly modulated by high-order optic flow patterns (e.g., spiral), yet a functional link to direct perception is lacking. Here, we applied electrical microstimulation to selectively manipulate population of MSTd neurons while macaques discriminated direction of rotation around line-of-sight (roll) or direction of linear-translation (heading), two tasks which were orthogonal in 3D spiral coordinate using a four-alternative-forced-choice paradigm. Microstimulation frequently biased animal’s roll perception towards coded labeled-lines of the artificial-stimulated neurons in either context with spiral or pure-rotation stimuli. Choice frequency was also altered between roll and translation flow-pattern. Our results provide direct causal-link evidence supporting that roll signals in MSTd, despite often mixed with translation signals, can be extracted by downstream areas for perception of rotation relative to gravity-vertical.

Distributed encoding of curvilinear self-motion across spiral optic flow patterns

Self-motion along linear paths without eye movements creates optic flow that radiates from the direction of travel (heading). Optic flow-sensitive neurons in primate brain area MSTd have been linked to linear heading perception, but the neural basis of more general curvilinear self-motion perception is unknown. The optic flow in this case is more complex and depends on the gaze direction and curvature of the path. We investigated the extent to which signals decoded from a neural model of MSTd predict the observer’s curvilinear self-motion. Specifically, we considered the contributions of MSTd-like units that were tuned to radial, spiral, and concentric optic flow patterns in “spiral space”. Self-motion estimates decoded from units tuned to the full set of spiral space patterns were substantially more accurate and precise than those decoded from units tuned to radial expansion. Decoding only from units tuned to spiral subtypes closely approximated the performance of the full model. Only the full decoding model could account for human judgments when path curvature and gaze covaried in self-motion stimuli. The most predictive units exhibited bias in center-of-motion tuning toward the periphery, consistent with neurophysiology and prior modeling. Together, findings support a distributed encoding of curvilinear self-motion across spiral space.

Cortical Motion Perception Emerges from Dimensionality Reduction with Evolved Spike-Timing-Dependent Plasticity Rules.

The nervous system is under tight energy constraints and must represent information efficiently. This is particularly relevant in the dorsal part of the medial superior temporal area (MSTd) in primates where neurons encode complex motion patterns to support a variety of behaviors. A sparse decomposition model based on a dimensionality reduction principle known as non-negative matrix factorization (NMF) was previously shown to account for a wide range of monkey MSTd visual response properties. This model resulted in sparse, parts-based representations that could be regarded as basis flow fields, a linear superposition of which accurately reconstructed the input stimuli. This model provided evidence that the seemingly complex response properties of MSTd may be a by-product of MSTd neurons performing dimensionality reduction on their input. However, an open question is how a neural circuit could carry out this function. In the current study, we propose a spiking neural network (SNN) model of MSTd based on evolved spike-timing-dependent plasticity and homeostatic synaptic scaling (STDP-H) learning rules. We demonstrate that the SNN model learns compressed and efficient representations of the input patterns similar to the patterns that emerge from NMF, resulting in MSTd-like receptive fields observed in monkeys. This SNN model suggests that STDP-H observed in the nervous system may be performing a similar function as NMF with sparsity constraints, which provides a test bed for mechanistic theories of how MSTd may efficiently encode complex patterns of visual motion to support robust self-motion perception.SIGNIFICANCE STATEMENT The brain may use dimensionality reduction and sparse coding to efficiently represent stimuli under metabolic constraints. Neurons in monkey area MSTd respond to complex optic flow patterns resulting from self-motion. We developed a spiking neural network model that showed MSTd-like response properties can emerge from evolving spike-timing-dependent plasticity with STDP-H parameters of the connections between then middle temporal area and MSTd. Simulated MSTd neurons formed a sparse, reduced population code capable of encoding perceptual variables important for self-motion perception. This model demonstrates that complex neuronal responses observed in MSTd may emerge from efficient coding and suggests that neurobiological plasticity, like STDP-H, may contribute to reducing the dimensions of input stimuli and allowing spiking neurons to learn sparse representations.

Estimating curvilinear self-motion from optic flow with a biologically inspired neural system* *This work was supported by The Office of Naval Research ONR N00014-18-1-2283.

Optic flow provides rich information about world-relative self-motion and is used by many animals to guide movement. For example, self-motion along linear, straight paths without eye movements, generates optic flow that radiates from a singularity that specifies the direction of travel (heading). Many neural models of optic flow processing contain heading detectors that are tuned to the position of the singularity, the design of which is informed by brain area MSTd of primate visual cortex that has been linked to heading perception. Such biologically inspired models could be useful for efficient self-motion estimation in robots, but existing systems are tailored to the limited scenario of linear self-motion and neglect sensitivity to self-motion along more natural curvilinear paths. The observer in this case experiences more complex motion patterns, the appearance of which depends on the radius of the curved path (path curvature) and the direction of gaze. Indeed, MSTd neurons have been shown to exhibit tuning to optic flow patterns other than radial expansion, a property that is rarely captured in neural models. We investigated in a computational model whether a population of MSTd-like sensors tuned to radial, spiral, ground, and other optic flow patterns could support the accurate estimation of parameters describing both linear and curvilinear self-motion. We used deep learning to decode self-motion parameters from the signals produced by the diverse population of MSTd-like units. We demonstrate that this system is capable of accurately estimating curvilinear path curvature, clockwise/counterclockwise sign, and gaze direction relative to the path tangent in both synthetic and naturalistic videos of simulated self-motion. Estimates remained stable over time while rapidly adapting to dynamic changes in the observer’s curvilinear self-motion. Our results show that coupled biologically inspired and artificial neural network systems hold promise as a solution for robust vision-based self-motion estimation in robots.

Longitudinal study of infants receiving extra motor stimulation, full-term control infants, and infants born preterm: High-density EEG analyses of cortical activity in response to visual motion.

Electroencephalography was used to investigate the effects of extrastimulation and preterm birth on the development of visual motion perception during early infancy. Infants receiving extra motor stimulation in the form of baby swimming, a traditionally raised control group, and preterm born infants were presented with an optic flow pattern simulating forward and reversed self‐motion and unstructured random visual motion before and after they achieved self‐produced locomotion. Extrastimulated infants started crawling earlier and displayed significantly shorter N2 latencies in response to visual motion than their full‐term and preterm peers. Preterm infants could not differentiate between visual motion conditions, nor did they significantly decrease their latencies with age and locomotor experience. Differences in induced activities were also observed with desynchronized theta‐band activity in all infants, but with more mature synchronized alpha–beta band activity only in extrastimulated infants after they had become mobile. Compared with the other infants, preterm infants showed more widespread desynchronized oscillatory activities at lower frequencies at the age of 1 year (corrected for prematurity). The overall advanced performance of extrastimulated infants was attributed to their enriched motor stimulation. The poorer responses in the preterm infants could be related to impairment of the dorsal visual stream that is specialized in the processing of visual motion.

Influence of Tactile Flow on Visual Heading Perception.

The integration of information from different sensory modalities is crucial for successful navigation through an environment. Among others, self-motion induces distinct optic flow patterns on the retina, vestibular signals and tactile flow, which contribute to determine traveled distance (path integration) or movement direction (heading). While the processing of combined visual-vestibular information is subject to a growing body of literature, the processing of visuo-tactile signals in the context of self-motion has received comparatively little attention. Here, we investigated whether visual heading perception is influenced by behaviorally irrelevant tactile flow. In the visual modality, we simulated an observer's self-motion across a horizontal ground plane (optic flow). Tactile self-motion stimuli were delivered by air flow from head-mounted nozzles (tactile flow). In blocks of trials, we presented only visual or tactile stimuli and subjects had to report their perceived heading. In another block of trials, tactile and visual stimuli were presented simultaneously, with the tactile flow within ±40° of the visual heading (bimodal condition). Here, importantly, participants had to report their perceived visual heading. Perceived self-motion direction in all conditions revealed a centripetal bias, i.e., heading directions were perceived as compressed toward straight ahead. In the bimodal condition, we found a small but systematic influence of task-irrelevant tactile flow on visually perceived headings as function of their directional offset. We conclude that tactile flow is more tightly linked to self-motion perception than previously thought.

Populations of local direction-selective cells encode global motion patterns generated by self-motion.

Self-motion generates visual patterns on the eye that are important for navigation. These optic flow patterns are encoded by the population of local direction–selective cells in the mouse retina, whereas in flies, local direction–selective T4/T5 cells are thought to be uniformly tuned. How complex global motion patterns can be computed downstream is unclear. We show that the population of T4/T5 cells in Drosophila encodes global motion patterns. Whereas the mouse retina encodes four types of optic flow, the fly visual system encodes six. This matches the larger number of degrees of freedom and the increased complexity of translational and rotational motion patterns during flight. The four uniformly tuned T4/T5 subtypes described previously represent a local subset of the population. Thus, a population code for global motion patterns appears to be a general coding principle of visual systems that matches local motion responses to modes of the animal’s movement.

Modeling Physiological Sources of Heading Bias from Optic Flow.

Human heading perception from optic flow is accurate for directions close to the straight-ahead and systematic biases emerge in the periphery (Cuturi and Macneilage, 2013; Sun et al., 2020). In pursuit of the underlying neural mechanisms, primate brain dorsal medial superior temporal (MSTd) area has been a focus because of its causal link with heading perception (Gu et al., 2012). Computational models generally explain heading sensitivity in individual MSTd neurons as a feedforward integration of motion signals from medial temporal (MT) area that resemble full-field optic flow patterns consistent with the preferred heading direction (Britten, 2008; Mineault et al., 2012). In the present simulation study, we quantified within the structure of this feedforward model how physiological properties of MT and MSTd shape heading signals. We found that known physiological tuning characteristics generally supported the accuracy of heading estimation, but not always. A weak-to-moderate overrepresentation of peripheral headings in MSTd garnered the highest accuracy and precision out of the models that we tested. The model also performed well when noise corrupted high proportions of the optic flow vectors. Such a peripheral MSTd model performed well when units possessed a range of receptive field (RF) sizes and were strongly direction tuned. Physiological biases in MT direction tuning toward the radial direction also supported heading estimation, but the tendency for MT preferred speed and RF size to scale with eccentricity did not. Our findings help elucidate the extent to which different physiological tuning properties influence the accuracy and precision of neural heading signals.

Disentangling of Local and Wide-Field Motion Adaptation.

Motion adaptation has been attributed in flying insects a pivotal functional role in spatial vision based on optic flow. Ongoing motion enhances in the visual pathway the representation of spatial discontinuities, which manifest themselves as velocity discontinuities in the retinal optic flow pattern during translational locomotion. There is evidence for different spatial scales of motion adaptation at the different visual processing stages. Motion adaptation is supposed to take place, on the one hand, on a retinotopic basis at the level of local motion detecting neurons and, on the other hand, at the level of wide-field neurons pooling the output of many of these local motion detectors. So far, local and wide-field adaptation could not be analyzed separately, since conventional motion stimuli jointly affect both adaptive processes. Therefore, we designed a novel stimulus paradigm based on two types of motion stimuli that had the same overall strength but differed in that one led to local motion adaptation while the other did not. We recorded intracellularly the activity of a particular wide-field motion-sensitive neuron, the horizontal system equatorial cell (HSE) in blowflies. The experimental data were interpreted based on a computational model of the visual motion pathway, which included the spatially pooling HSE-cell. By comparing the difference between the recorded and modeled HSE-cell responses induced by the two types of motion adaptation, the major characteristics of local and wide-field adaptation could be pinpointed. Wide-field adaptation could be shown to strongly depend on the activation level of the cell and, thus, on the direction of motion. In contrast, the response gain is reduced by local motion adaptation to a similar extent independent of the direction of motion. This direction-independent adaptation differs fundamentally from the well-known adaptive adjustment of response gain according to the prevailing overall stimulus level that is considered essential for an efficient signal representation by neurons with a limited operating range. Direction-independent adaptation is discussed to result from the joint activity of local motion-sensitive neurons of different preferred directions and to lead to a representation of the local motion direction that is independent of the overall direction of global motion.