Visual Motion Cues Research Articles

Divergent Visuomotor Strategies in Teleosts: Neural Circuit Mechanisms in Zebrafish and Danionella cerebrum.

Larval DC exhibit faster swimming than ZF , matching the direction of visual motion. DC execute OMR in smooth, curved swimming patterns, interspersed with sharp directional turns. ZF and DC share similar visuomotor neural architecture, recruiting pretectal and hindbrain regions. ZF and DC demonstrate lateralized encoding of turns, particularly in medial hindbrain neurons. Larval Danionella cerebrum respond to global visual motion cues in smooth, low-angle swimming patterns, interspersed with sharp directional turns, swimming consistently faster than zebrafish. Fouke et al. use behavioral tracking of freely moving and head fixed fish to reveal an evolutionarily conserved visuomotor neural architecture transforming visual motion cues into species-specific locomotor behaviors.

Monkeys engage in visual simulation to solve complex problems.

Visual simulation-i.e., using internal reconstructions of the world to experience potential future versions of events that are not currently happening-is among the most sophisticated capacities of the human mind. But is this ability in fact uniquely human? To answer this question, we tested monkeys on a series of experiments involving the "Planko" game, which we have previously used to evoke visual simulation in human participants. We found that monkeys were able to successfully play the game using a simulation strategy, predicting the trajectory of a ball through a field of planks while demonstrating a level of accuracy and behavioral signatures comparable with those of humans. Computational analyses further revealed that the monkeys' strategy while playing Planko aligned with a recurrent neural network (RNN) that approached the task using a spontaneously learned simulation strategy. Finally, we carried out awake functional magnetic resonance imaging while monkeys played Planko. We found activity in motion-sensitive regions of the monkey brain during hypothesized simulation periods, even without any perceived visual motion cues. This neural result closely mirrors previous findings from human research, suggesting a shared mechanism of visual simulation across species. Taken together, these findings challenge traditional views of animal cognition, proposing that nonhuman primates possess a complex cognitive landscape, capable of invoking imaginative and predictive mental experiences to solve complex everyday problems.

Counterillumination reduces bites by Great White sharks.

In the open ocean, achieving camouflage is complicated by the fact that the downwelling light is generally much brighter than the upwelling light, which means that any object, even if its ventral surface is white due to countershading, will appear as a dark silhouette when viewed from below.1,2,3 To overcome this, many marine species employ counterillumination, whereby light is emitted from photophores on their ventral surface to replace the downwelling light blocked by their body.4,5,6 However, only a single behavioral study has tested the efficacy of counterillumination as an anti-predation strategy.7 Counterillumination is predicted to be particularly useful against predators that have poor visual acuity and lack color vision,8 like the Great White shark (Carcharodon carcharias), the species responsible for most human shark-bite fatalities globally.9 Here, we take inspiration from nature to show that counterillumination can prevent Great White sharks from attacking artificial seal decoys. Using seal decoys fitted with LED lights and towed behind a boat, we explored the efficiency of different light configurations on the deterrence effect, showing that visual shape and motion cues are critical for prey recognition by Great White sharks. Counterillumination that is brighter than the background is most effective in deterring sharks, implying that, in this context, counterillumination works through disruptive camouflage rather than background matching. Our results reveal the importance of a dark silhouette against a lighter background in predatory behavior in Great White sharks and that altering the silhouette may form the basis of new non-invasive shark deterrent technology to protect human life. VIDEO ABSTRACT.

Bilateral interactions of optic-flow sensitive neurons coordinate course control in flies

Animals rely on compensatory actions to maintain stability and navigate their environment efficiently. These actions depend on global visual motion cues known as optic-flow. While the optomotor response has been the traditional focus for studying optic-flow compensation in insects, its simplicity has been insufficient to determine the role of the intricate optic-flow processing network involved in visual course control. Here, we reveal a series of course control behaviours in Drosophila and link them to specific neural circuits. We show that bilateral electrical coupling of optic-flow-sensitive neurons in the fly’s lobula plate are required for a proper course control. This electrical interaction works alongside chemical synapses within the HS-H2 network to control the dynamics and direction of turning behaviours. Our findings reveal how insects use bilateral motion cues for navigation, assigning a new functional significance to the HS-H2 network and suggesting a previously unknown role for gap junctions in non-linear operations.

Visual Cue Effects on a Classification Accuracy Estimation Task in Immersive Scatterplots.

Immersive visualization in virtual reality (VR) allows us to exploit visual cues for perception in 3D space, yet few existing studies have measured the effects of visual cues. Across a desktop monitor and a head-mounted display (HMD), we assessed scatterplot designs which vary their use of visual cues-motion, shading, perspective (graphical projection), and dimensionality-on two sets of data. We conducted a user study with a summary task in which 32 participants estimated the classification accuracy of an artificial neural network from the scatterplots. With Bayesian multilevel modeling, we capture the intricate visual effects and find that no cue alone explains all the variance in estimation error. Visual motion cues generally reduce participants' estimation error; besides this motion, using other cues may increase participants' estimation error. Using an HMD, adding visual motion cues, providing a third data dimension, or showing a more complicated dataset leads to longer response times. We speculate that most visual cues may not strongly affect perception in immersive analytics unless they change people's mental model about data. In summary, by studying participants as they interpret the output from a complicated machine learning model, we advance our understanding of how to use the visual cues in immersive analytics.

Neck stabilization through sensory integration of vestibular and visual motion cues.

To counteract gravity, trunk motion, and other perturbations, the human head-neck system requires continuous muscular stabilization. In this study, we combine a musculoskeletal neck model with models of sensory integration (SI) to unravel the role of vestibular, visual, and muscle sensory cues in head-neck stabilization and relate SI conflicts and postural instability to motion sickness. A 3D multisegment neck model with 258 Hill-type muscle elements was extended with postural stabilization using SI of vestibular (semicircular and otolith) and visual (rotation rate, verticality, and yaw) cues using the multisensory observer model (MSOM) and the subjective vertical conflict model (SVC). Dynamic head-neck stabilization was studied using empirical datasets, including 6D trunk perturbations and a 4 m/s2 slalom drive inducing motion sickness. Recorded head translation and rotation are well matched when using all feedback loops with MSOM or SVC or assuming perfect perception. A basic version of the model, including muscle, but omitting vestibular and visual perception, shows that muscular feedback can stabilize the neck in all conditions. However, this model predicts excessive head rotations in conditions with trunk rotation and in the slalom. Adding feedback of head rotational velocity sensed by the semicircular canals effectively reduces head rotations at mid-frequencies. Realistic head rotations at low frequencies are obtained by adding vestibular and visual feedback of head rotation based on the MSOM or SVC model or assuming perfect perception. The MSOM with full vision well captures all conditions, whereas the MSOM excluding vision well captures all conditions without vision. The SVC provides two estimates of verticality, with a vestibular estimate SVCvest, which is highly effective in controlling head verticality, and an integrated vestibular/visual estimate SVCint which can complement SVCvest in conditions with vision. As expected, in the sickening drive, SI models imprecisely estimate verticality, resulting in sensory conflict and postural instability. The results support the validity of SI models in postural stabilization, where both MSOM and SVC provide credible results. The results in the sickening drive show imprecise sensory integration to enlarge head motion. This uniquely links the sensory conflict theory and the postural instability theory in motion sickness causation.

Stimulus Onset Asynchrony Affects Weighting-related Event-related Spectral Power in Self-motion Perception.

Self-motion perception relies primarily on the integration of the visual, vestibular, proprioceptive, and somatosensory systems. There is a gap in understanding how a temporal lag between visual and vestibular motion cues affects visual-vestibular weighting during self-motion perception. The beta band is an index of visual-vestibular weighting, in that robust beta event-related synchronization (ERS) is associated with visual weighting bias, and robust beta event-related desynchronization is associated with vestibular weighting bias. The present study examined modulation of event-related spectral power during a heading judgment task in which participants attended to either visual (optic flow) or physical (inertial cues stimulating the vestibular, proprioceptive and somatosensory systems) motion cues from a motion simulator mounted on a MOOG Stewart Platform. The temporal lag between the onset of visual and physical motion cues was manipulated to produce three lag conditions: simultaneous onset, visual before physical motion onset, and physical before visual motion onset. There were two main findings. First, we demonstrated that when the attended motion cue was presented before an ignored cue, the power of beta associated with the attended modality was greater than when visual-vestibular cues were presented simultaneously or when the ignored cue was presented first. This was the case for beta ERS when the visual-motion cue was attended to, and beta event-related desynchronization when the physical-motion cue was attended to. Second, we tested whether the power of feature-binding gamma ERS (demonstrated in audiovisual and visual-tactile integration studies) increased when the visual-vestibular cues were presented simultaneously versus with temporal asynchrony. We did not observe an increase in gamma ERS when cues were presented simultaneously, suggesting that electrophysiological markers of visual-vestibular binding differ from markers of audiovisual and visual-tactile integration. All event-related spectral power reported in this study were generated from dipoles projecting from the left and right motor areas, based on the results of Measure Projection Analysis.

Congruent visual cues speed dynamic motor adaptation

We demonstrate that adaptation to novel dynamics is stronger when additional online visual cues that are congruent with the dynamics are presented during adaptation, compared with either a constant or incongruent visual cue. This effect was found regardless of whether a bidirectional or unidirectional velocity-dependent force field was presented to the participants. We propose that this effect might arise through the inclusion of this additional visual cue information within the state estimation process.

Vestibular and visual brain areas in the medial cortex of the human brain.

Self-motion perception involves an interaction between vestibular and visual brain regions. In the lateral brain, it includes the parietoinsular vestibular cortex and the posterior insular cortex. In the medial cortex, the cingulate sulcus visual (CSv) area is known to process visual-vestibular cues. Here, we show that the vestibular-visual network of the medial cortex extends beyond area CSv. We examined brain activations of 36 healthy right-handed participants by functional magnetic resonance imaging (fMRI) during stimulation with caloric vestibular, thermal, or visual motion cues. Consistent with previous research, we found that area CSv responded to both vestibular and visual cues but not to thermal cues. Moreover, the V6 complex and the precuneus motion (PcM) area responded primarily to (laminar-translational) visual motion cues. However, we also observed a region inferior to CSv within the pericallosal sulcus (vicinity of anterior retrosplenial) that primarily responded to vestibular cues. This vestibular pericallosal sulcus (vPCS) region did not respond to either visual or thermal cues. It was also distinct from a more posterior motion-sensitive region in the retrosplenial complex (mRSC) that responded to (radial) visual motion but not to vestibular and thermal cues. Together, our results suggest that the vestibular-visual network in the medial cortex not only includes areas CSv, PcM, and the V6 complex but also two additional brain regions adjacent to the callosum. These two brain regions exhibit similarities in terms of their locations and responses to vestibular and visual cues with self-motion-related brain regions recently described in nonhuman primates.NEW & NOTEWORTHY Self-motion perception involves several vestibular and visual cortical regions. Within the medial cortex, the cingulate sulcus visual (CSv) area, the precuneus motion (PcM) area, and the V6 complex respond selectively to self-motion cues. Here, we show that vestibular information is also processed in the pericallosal sulcus (vPCS), whereas (radial) visual motion information is associated with activation in the retrosplenial cortex (mRSC).

Social Inference from Relational Visual Information

Humans easily make social evaluations from visual input, such as recognizing a social interaction or deciding who is a friend and who is a foe. Prior efforts to model this ability have suggested that visual features and motion cues alone cannot account for human performance in social tasks, like distinguishing between helping and hindering interactions. On the other hand, generative Bayesian inference models, which make predictions based on simulations of agents’ social or physical goals, accurately predict human judgments, but are computationally very expensive and often impossible to implement in natural visual stimuli. Inspired by developmental work, we hypothesize that introducing inductive biases in visual models would allow them to make more human-like social judgments. Specifically, in this study, we investigate if relational representations of visual stimuli using graph neural networks can predict human social interaction judgments. We use the PHASE dataset, consisting of 2D animations of two agents and two objects resembling real-life social interactions, rated as friendly, neutral, or adversarial. We propose a graph neural networks (GNN) based architecture, that takes in graph representations of a video and predicts the relationship between the agents. We collected human ratings for each of the 400 videos and found that our GNN model aligns with human judgments significantly better than a baseline visual model with the same visual/motion information but without the graph structure (79% vs 62% prediction of human judgements). Intriguingly, explicitly adding relational information to the baseline model does not improve its performance, suggesting that graphical representations in particular are important to modeling human judgements. Taken together, these results suggest that relational graphical representations of visual information can help artificial vision systems make more human-like social judgments without incurring the computational cost of Bayesian models, and provide insights into the computations that humans employ while making visual social judgments.

Metacognitive understanding of visual motion cues to intentionality

Metacognitive understanding of visual motion cues to intentionality

Combination of visuo-tactile and visuo-vestibular correlations in illusory body ownership and self-motion sensations.

Previous studies have shown that illusory ownership over a mannequin's body can be induced through synchronous visuo-tactile stimulation as well as through synchronous visuo-vestibular stimulation. The current study aimed to elucidate how three-way combinations of correlated visual, tactile and vestibular signals contribute to the senses of body ownership and self-motion. Visuo-tactile temporal congruence was manipulated by touching the mannequin's body and the participant's unseen real body on the trunk with a small object either synchronously or asynchronously. Visuo-vestibular temporal congruence was manipulated by synchronous or asynchronous presentation of a visual motion cue (the background rotating around the mannequin in one direction) and galvanic stimulation of the vestibular nerve generating a rotation sensation (in the same direction). The illusory experiences were quantified using a questionnaire; threat-evoked skin-conductance responses (SCRs) provided complementary indirect physiological evidence for the illusion. Ratings on the illusion questionnaire statement showed significant main effects of synchronous visuo-vestibular and synchronous visuo-tactile stimulations, suggesting that both of these pairs of bimodal correlations contribute to the ownership illusion. Interestingly, visuo-tactile synchrony dominated because synchronous visuo-tactile stimulation combined with asynchronous visuo-vestibular stimulation elicited a body ownership illusion of similar strength as when both bimodal combinations were synchronous. Moreover, both visuo-tactile and visuo-vestibular synchrony were associated with enhanced self-motion perception; self-motion sensations were even triggered when visuo-tactile synchrony was combined with visuo-vestibular asynchrony, suggesting that ownership enhanced the relevance of visual information as a self-motion cue. Finally, the SCR results suggest that synchronous stimulation of either modality pair led to a stronger illusion compared to the asynchronous conditions. Collectively, the results suggest that visuo-tactile temporal correlations have a stronger influence on body ownership than visuo-vestibular correlations and that ownership boosts self-motion perception. We present a Bayesian causal inference model that can explain how visuo-vestibular and visuo-tactile information are combined in multisensory own-body perception.

Motion vision: Drosophila neural pathways that go with the visual flow

Processing visual motion cues to interpret self-motion, the movement of others, and the environment's structure is vital to all animals, whether prey or predator. A new study in Drosophila identifies multiple pathways likely contributing to visual motion-dependent computations and behaviors.

Prey interactions in tiger sharks: Accounting for visual perception in animal-borne cameras

The ‘evolutionary arms race’ between predator and prey is responsible for many physiological, morphological, and behavioural adaptations and some of the most compelling examples relate to the generation and detection of visual signals. Better understanding of adaptations can arise from approaching predator-prey dynamics from the point of view of the animals themselves, such as through animal-borne cameras. As we learn more about an animal's visual system, we can take this further and analyse visual scenes through their eyes. In this study video cameras mounted on tiger sharks were used to analyse motion cues of sea turtles using two-dimensional motion detector (2DMD) models. Sharks were also equipped with tri-axial motion-sensors to compare visual motion cues to behaviour, using the behavioural metrics of overall dynamic body acceleration (ODBA), path tortuosity, tailbeat frequency and tailbeat acceleration signal amplitude. When comparing the behaviours of sharks in the minute before and after interactions with sea turtles, we found lower ODBA of sharks associated with sea turtles that produced greater visual motion cues. These turtles also invoked swimming by sharks that was more tortuous than when no turtle was encountered. Together, these behaviours suggest that tiger sharks may potentially stalking turtles that produced greater motion cues. Many sea turtles did not stand out from the background motion suggesting they remain motionless as a form of camouflage. This study provides insights into the use of visual cues in prey identification by tiger sharks and the camouflage strategies used by sea turtles to avoid predation.

Resilience of visually guided weight shifting to a proprioceptive perturbation depends on the complexity of the guidance stimulus

BackgroundWhole-body tracking of visual motion cues is used in balance training to improve weight shifting ability in old age and sports. Research questionHow tracking of a complex (pink noise) and a periodic visual target motion during anteroposterior weight shifting affects postural and muscle responses to unilateral hip vibration. MethodsTwenty-six participants performed 160 anteroposterior weight shifting cycles while tracking the vertical motion of a visual target, concurrently receiving Center of Pressure (CoP) feedback. They were randomly divided to groups; (a) the Constant group tracked a visual target motion constructed by 3 sinusoids of different amplitude, and (b) the Pink group tracked a complex visual target motion constructed by a pink noise generation process. Between the 60th and the 120th cycle, vibration was applied to the right gluteus medius, introducing a sideways CoP deviation. CoP displacement and electromyographic (EMG) responses of soleus, tibialis anterior and peroneus longus were recorded and summarized in blocks of 3 cycles. ResultsSideways CoP deviation induced at the onset/offset of unilateral hip vibration was smaller for the Pink than the Constant group. The Pink group demonstrated greater tibialis anterior and peroneus longus EMG activity around the most anterior sway peak while soleus EMG was similar for the two groups. Both groups successfully coupled weight shifting amplitude to the target motion, but the Pink group tracked the target motion with a greater delay compared to the Constant group. SignificanceWhole body tracking of complex visual motions evokes perception-based action and increases ankle muscle co-activation making sway more resilient to a proprioceptive perturbation induced by unilateral hip vibration. Complex visual guidance motions should be considered when designing balance rehabilitation regimes, aiming at improving weight shifting ability and dynamic balance control.

Posture Deficits and Recovery After Unilateral Vestibular Loss: Early Rehabilitation and Degree of Hypofunction Matter.

Postural instability and balance impairment are disabling symptoms in patients with acute unilateral peripheral vestibular hypofunction (UVH). Vestibular rehabilitation (VR) is known to improve the vestibular compensation process, but (1) its effect on posture recovery remains poorly understood, (2) little is known about when VR must be done, and (3) whether the degree of vestibular loss matters is uncertain. We analyzed posture control under static (stable support) and dynamic (unstable support) postural tasks performed in different visual conditions [eye open (EO); eyes closed (EC); and optokinetic stimulation] using dynamic posturography. Non-linear analyses of the postural performance (wavelet transform, diffusion analysis, and fractal analysis) were performed in two groups of patients with UVH subjected to the same VR program based on the unidirectional rotation paradigm and performed either early (first 2 weeks) or later (fifth to the sixth week) after vertigo attack. Distribution of the angular horizontal vestibulo-ocular reflex (aVOR) gain values recorded on the hypofunction side before rehabilitation differentiated two distinct sub-groups (cluster analysis) with aVOR gains below or above 0.20. The postural performance of the four sub-groups of patients with UVH (early rehabilitation with aVOR gain <0.20: n = 25 or gain >0.20: n = 19; late rehabilitation with aVOR gain <0.20: n = 15 or gain >0.20: n = 10) tested before VR showed significantly altered postural parameters compared with healthy controls. Greater instability, higher energy to control posture, larger sway without feedback corrections, and lower time of automatic control of posture were observed in static conditions. The four sub-groups recovered near-normal postural performance after VR in the EO and EC conditions, but still exhibited altered postural performance with optokinetic stimulation. In dynamic posturography conditions and before VR, the percentage of patients able to perform the postural tasks with EC and optokinetic stimulation was significantly lower in the two sub-groups with aVOR gain <0.20. After VR, the improvement of the postural parameters depended on the stage of rehabilitation and the degree of vestibular hypofunction. The best balance function recovery was found in the sub-group with early VR and pre-rehabilitation aVOR gain above 0.20, the worst in the sub-group with late rehabilitation and aVOR gain below 0.20. These differences were seen when the vestibular input remains the main sensory cue to control balance, that is, on unstable support without vision or altered visual motion cues. These findings extend to dynamic balance recovery the crucial roles of early rehabilitation and degree of vestibular hypofunction which we have already highlighted for vestibulo-ocular reflex recovery.

How multisensory neurons solve causal inference

Sitting in a static railway carriage can produce illusory self-motion if the train on an adjoining track moves off. While our visual system registers motion, vestibular signals indicate that we are stationary. The brain is faced with a difficult challenge: is there a single cause of sensations (I am moving) or two causes (I am static, another train is moving)? If a single cause, integrating signals produces a more precise estimate of self-motion, but if not, one cue should be ignored. In many cases, this process of causal inference works without error, but how does the brain achieve it? Electrophysiological recordings show that the macaque medial superior temporal area contains many neurons that encode combinations of vestibular and visual motion cues. Some respond best to vestibular and visual motion in the same direction ("congruent" neurons), while others prefer opposing directions ("opposite" neurons). Congruent neurons could underlie cue integration, but the function of opposite neurons remains a puzzle. Here, we seek to explain this computational arrangement by training a neural network model to solve causal inference for motion estimation. Like biological systems, the model develops congruent and opposite units and recapitulates known behavioral and neurophysiological observations. We show that all units (both congruent and opposite) contribute to motion estimation. Importantly, however, it is the balance between their activity that distinguishes whether visual and vestibular cues should be integrated or separated. This explains the computational purpose of puzzling neural representations and shows how a relatively simple feedforward network can solve causal inference.

Interfering Embodiment Effects on Chinese “Transfer Verbs”

Abstract This research aims to explore the processing of embodied meaning during the comprehension of Chinese transfer verbs which is different from the typical structure of transfer verbs in English and other Indo-European languages. An Action-sentence Compatibility Effect (ACE) paradigm was used, in which participants were asked to read sentences describing a transfer verb either away from (At the court, a player throws tennis ball to opposite side) or toward themselves (At the court, a player throws tennis ball to my side). Following the transfer verb, a visual motion cue appeared on the screen after one of the three stimulus onset asychrony (SOA), prompting participants to move their hand either away from or toward themselves by pressing a button. The results showed that under short SOAs (cue presented 100 ms or 200 ms after the verb onset), interference occurred in the matching conditions. After larger delaying of the cue (350 ms), facilitation emerged in matching conditions. The results reflect special features in describing motion events by using Chinese transfer verbs, providing evidence that the comprehension of transfer-verb sentences in Mandarin activates the sensory-motor systems of our body, either interfering or facilitating a motor response performed in parallel.

The cingulate oculomotor cortex

Knowledge about the relevance and extent of human eye movement control in the cingulate cortex to date is very limited. Experiments in non-human primates brought about evidence for a potentially central role of the dorsal bank of the cingulate sulcus in saccadic eye movements. In humans, a putative cingulate eye field (CEF) in the same region has been proposed; however, its function and location still remain controversial. Another area in the posterior cingulate cortex, the cingulate sulcus visual area (CSv), has been shown to respond to visual motion cues and also ocular motor tasks. In this study we used multi-band neuroimaging (n = 46) to comprehensively characterize oculomotor responses along the entire cingulate cortex during the most common types of eye movements.We were able to robustly localize the CEF to the anterior portion of the midcingulate gyrus. The region gave responses during all oculomotor tasks and is embedded within the ventral attention network. Area CSv, which is located in the anterior portion of the posterior cingulate gyrus, on the other hand responded to smooth pursuit and optokinetic nystagmus only. It likewise represents a node within the ventral attention network but at the same time seems to be a distinctive part of the somatomotor network. Our findings support an executive role of the CEF, suggesting a cognitive control function in maintaining and adapting different kinds of eye movements. CSv on the other hand might be an interface for relaying oculomotor, visual motion and broad sensory signals related to self-motion.

Newborns\u2019 sensitivity to speed changes as a building block for animacy perception

The human visual system can discriminate between animate beings vs. inanimate objects on the basis of some kinematic cues, such as starting from rest and speed changes by self-propulsion. The ontogenetic origin of such capability is still under debate. Here we investigate for the first time whether newborns manifest an attentional bias toward objects that abruptly change their speed along a trajectory as contrasted with objects that move at a constant speed. To this end, we systematically manipulated the motion speed of two objects. An object that moves with a constant speed was contrasted with an object that suddenly increases (Experiment 1) or with one that suddenly decreases its speed (Experiment 2). When presented with a single speed change, newborns did not show any visual preference. However, newborns preferred an object that abruptly increases and then decreases its speed (Experiment 3), but they did not show any visual preference for the reverse sequence pattern (Experiment 4). Overall, results are discussed in line with the hypothesis of the existence of attentional biases in newborns that trigger their attention towards some visual cues of motion that characterized animate perception in adults.