Local Motion Cues Research Articles

Audiovisual correspondence facilitates the visual search for biological motion

Hearing synchronous sounds may facilitate the visual search for the concurrently changed visual targets. Evidence for this audiovisual attentional facilitation effect mainly comes from studies using artificial stimuli with relatively simple temporal dynamics, indicating a stimulus-driven mechanism whereby synchronous audiovisual cues create a salient object to capture attention. Here, we investigated the crossmodal attentional facilitation effect on biological motion (BM), a natural, biologically significant stimulus with complex and unique dynamic profiles. We found that listening to temporally congruent sounds, compared with incongruent sounds, enhanced the visual search for BM targets. More intriguingly, such a facilitation effect requires the presence of distinctive local motion cues (especially the accelerations in feet movement) independent of the global BM configuration, suggesting a crossmodal mechanism triggered by specific biological features to enhance the salience of BM signals. These findings provide novel insights into how audiovisual integration boosts attention to biologically relevant motion stimuli and extend the function of a proposed life detection system driven by local kinematics of BM to multisensory life motion perception.

Hybrid Motion Representation Learning for Prediction From Raw Sensor Data

Motion prediction from raw LiDAR sensor data has drawn increasing attention and led to a surge of studies following two main paradigms. One paradigm is global motion paradigm, which simultaneously detects objects from point clouds and predicts the trajectories of each object in the future. The other paradigm is local motion paradigm, which directly performs dense motion prediction pointwisely. We observe that global motion prediction can benefit from local motion representation, since it contains rich local displacement contexts that are not explicitly exploited in global motion prediction. Correspondingly, local motion prediction can benefit from global motion representation, since it provides object contexts to improve prediction consistency inside an object. However, the complement of these two motion representations has not fully explored in the literature. To this end, we propose Hybrid Motion Representation Learning (HyMo), a unified framework to address the problem of motion prediction by making the best of both global and local motion cues. We have conducted extensive experiments on nuScenes dataset. The experimental results demonstrate that the learned hybrid motion representation achieves state-of-the-art performance on both global and local motion prediction tasks.

Brain Networks Processing Temporal Information in Dynamic Facial Expressions.

This fMRI study examines the role of local and global motion information in facial movements during exposure to novel dynamic face stimuli. We found that synchronous expressions distinctively engaged medial prefrontal areas in the rostral and caudal sectors of anterior cingulate cortex (r/cACC) extending to inferior supplementary motor areas, as well as motor cortex and bilateral superior frontal gyrus (global temporal-spatial processing). Asynchronous expressions in which one part of the face unfolded before the other activated more the right superior temporal sulcus (STS) and inferior frontal gyrus (local temporal-spatial processing). These differences in temporal dynamics had no effect on visual face-responsive areas. Dynamic causal modeling analysis further showed that processing of asynchronous expression features was associated with a differential information flow, centered on STS, which received direct input from occipital cortex and projected to the amygdala. Moreover, STS and amygdala displayed selective interactions with cACC where the integration of both local and global motion cues could take place. These results provide new evidence for a role of local and global temporal dynamics in emotional expressions, extracted in partly separate brain pathways. Importantly, we show that dynamic expressions with synchronous movement cues may distinctively engage brain areas responsible for motor execution of expressions.

Detection of deceptive motions in rugby from visual motion cues.

Frequently, in rugby, players incorporate deceptive motions (e.g., a side-step) in order to pass their opponent. Previous works showed that expert defenders are more efficient in detecting deceptive motions. Performance was shown to be correlated with the evolution of the center of gravity of the attacker, suggesting that experts may rely on global motion cues. This study aims at investigating whether a representation of center of gravity can be useful for training purposes, by using this representation alone or by combining it with the local motion cues given by body parts. We designed an experiment in virtual reality to control the motion cues available to the defenders. Sixteen healthy participants (seven experts and nine novices) acted as defenders while a virtual attacker approached. Participants completed two separate tasks. The first was a time occlusion perception task, occlusion after 100ms, 200ms or 300ms after the initial change in direction, thereafter participants indicated the passing direction of the attacker. The second was a perception-action task, participants were instructed to intercept the oncoming attacker by displacing medio-laterally. The attacker performed either a non-deceptive motion, directly toward the final passing direction or a deceptive motion, initially toward a false direction before quickly reorienting to the true direction. There was a main effect of expertise, appearance, cut off times and motion on correct responses during both tasks. There was an interaction between visual appearance and expertise, and between motion type and expertise during the perception task, however, this interaction was not present during the perception-action task. We observed that experts maintained superiority in the perception of deceptive motion; however when the visual appearance is reduced to global motion alone the difference between novices and experts is reduced. We further explore the interactions and discuss the effects observed for the visual appearance and expertise.

Sequential Nonlinear Filtering of Local Motion Cues by Global Motion Circuits

Many animals guide their movements using optic flow, the displacement of stationary objects across the retina caused by self-motion. How do animals selectively synthesize a global motion pattern from its local motion components? To what extent does this feature selectivity rely on circuit mechanisms versus dendritic processing? Here we used invivo calcium imaging to identify pre- and postsynaptic mechanisms for processing local motion signals in global motion detection circuits in Drosophila. Lobula plate tangential cells (LPTCs) detect global motion by pooling input from local motion detectors, T4/T5 neurons. We show that T4/T5 neurons suppress responses to adjacent local motion signals whereas LPTC dendrites selectively amplify spatiotemporal sequences of local motion signals consistent with preferred global patterns. We propose that sequential nonlinear suppression and amplification operations allow optic flow circuitry to simultaneously prevent saturating responses to local signals while creating selectivity for global motion patterns critical to behavior.

The effects of high-frequency transcranial random noise stimulation (hf-tRNS) on global motion processing: An equivalent noise approach

BackgroundHigh frequency transcranial random noise stimulation (hf-tRNS) facilitates performance in several perceptual and cognitive tasks, however, little is known about the underlying modulatory mechanisms. ObjectiveIn this study we compared the effects of hf-tRNS to those of anodal and cathodal tDCS in a global motion direction discrimination task. An equivalent noise (EN) paradigm was used to assess how hf-tRNS modulates the mechanisms underlying local and global motion processing. MethodMotion coherence threshold and slope of the psychometric function were estimated using an 8AFC task in which observers had to discriminate the motion direction of a random dot kinematogram presented either in the left or right visual hemi-field. During the task hf-tRNS, anodal and cathodal tDCS were delivered over the left hMT+. In a subsequent experiment we implemented an EN paradigm in order to investigate the effects of hf-tRNS on the mechanisms involved in visual motion integration (i.e., internal noise and sampling). Resultshf-tRNS reduced the motion coherence threshold but did not affect the slope of the psychometric function, suggesting no modulation of stimulus discriminability. Anodal and cathodal tDCS did not produce any modulatory effects. EN analysis in the last experiment found that hf-tRNS modulates sampling but not internal noise, suggesting that hf-tRNS modulates the integration of local motion cues. Conclusionhf-tRNS interacts with the output neurons tuned to directions near to the directional signal, incrementing the signal-to-noise ratio and the pooling of local motion cues and thus increasing the sensitivity for global moving stimuli.

Collision Avoidance Behavior between Walkers: Global and Local Motion Cues.

Daily activities require agents to interact with each other, such as during collision avoidance. The nature of visual information that is used for a collision free interaction requires further understanding. We aim to manipulate the nature of visual information in two forms, global and local information appearances. Sixteen healthy participants navigated towards a target in an immersive computer-assisted virtual environment (CAVE) using a joystick. A moving passive obstacle crossed the participant's trajectory perpendicularly at various pre-defined risks of collision distances. The obstacle was presented with one of five virtual appearances, associated to global motion cues (i.e., a cylinder or a sphere), or local motion cues (i.e., only the legs or the trunk). A full body virtual walker, showing both local and global motion cues, used as a reference condition. The final crossing distance was affected by the global motion appearances, however, appearance had no qualitative effect on motion adaptations. These findings contribute towards further understanding what information people use when interacting with others.

Video pose estimation with global motion cues

This paper addresses the problem of pose estimation in video sequences in which human pose changes drastically over time. Popular strategies for video pose estimation first yield multiple pose candidates for each frame and then achieve consistent pose estimation by enforcing temporal constraints across frames. To enrich pose candidates, previous methods typically employ local motion cues to propagate pose detections to adjacent frames. Reasonable pose proposals can be achieved only when the local motion estimation is accurate and good detections exist among adjacent frames, both of which are hard to be satisfied under drastic human pose changes. In this paper, we propose to propagate pose detections to entire video sequence through global motion cues which provide a long term holistic non-rigid motion transformation for the given video. We exploit the temporal continuity of both single parts and part pairs in the inference over a spatio-temporal model to stitch the reasonable trajectory fragments for each part and obtain the final pose estimation. Experimental results demonstrate remarkable performance improvement in comparison with the state-of-the-art methods.

Development of Three-Dimensional Perception in Human Infants

The play of light on the retina contains multiple sources of information about the three-dimensional (3D) structure of the world. Some of the best information is derived from differencing operations that act on the images that result from the two eyes' laterally displaced vantage points. Other information is available in systematic retinal patterns of local texture and motion cues. This article describes what is currently known about the development of sensitivity to these binocular and monocular cues for depth in human infants, and it places the results in the context of what is known about the underlying neural mechanisms from work in nonhuman primates and human neuroimaging studies.

The contribution of dynamic visual cues to audiovisual speech perception

Seeing a speaker's facial gestures can significantly improve speech comprehension, especially in noisy environments. However, the nature of the visual information from the speaker's facial movements that is relevant for this enhancement is still unclear. Like auditory speech signals, visual speech signals unfold over time and contain both dynamic configural information and luminance-defined local motion cues; two information sources that are thought to engage anatomically and functionally separate visual systems. Whereas, some past studies have highlighted the importance of local, luminance-defined motion cues in audiovisual speech perception, the contribution of dynamic configural information signalling changes in form over time has not yet been assessed. We therefore attempted to single out the contribution of dynamic configural information to audiovisual speech processing. To this aim, we measured word identification performance in noise using unimodal auditory stimuli, and with audiovisual stimuli. In the audiovisual condition, speaking faces were presented as point light displays achieved via motion capture of the original talker. Point light displays could be isoluminant, to minimise the contribution of effective luminance-defined local motion information, or with added luminance contrast, allowing the combined effect of dynamic configural cues and local motion cues. Audiovisual enhancement was found in both the isoluminant and contrast-based luminance conditions compared to an auditory-only condition, demonstrating, for the first time the specific contribution of dynamic configural cues to audiovisual speech improvement. These findings imply that globally processed changes in a speaker's facial shape contribute significantly towards the perception of articulatory gestures and the analysis of audiovisual speech.

Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling.

We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling. We demonstrate our technique in cases where past methods fail (non-translating cameras and dynamic scenes). Our technique is applicable to single images as well as videos. For videos, we use local motion cues to improve the inferred depth maps, while optical flow is used to ensure temporal depth consistency. For training and evaluation, we use a Kinect-based system to collect a large data set containing stereoscopic videos with known depths. We show that our depth estimation technique outperforms the state-of-the-art on benchmark databases. Our technique can be used to automatically convert a monoscopic video into stereo for 3D visualization, and we demonstrate this through a variety of visually pleasing results for indoor and outdoor scenes, including results from the feature film Charade.

Local and global aspects of biological motion perception in children born at very low birth weight

Biological motion perception can be assessed using a variety of tasks. In the present study, 8- to 11-year-old children born prematurely at very low birth weight (<1500 g) and matched, full-term controls completed tasks that required the extraction of local motion cues, the ability to perceptually group these cues to extract information about body structure, and the ability to carry out higher order processes required for action recognition and person identification. Preterm children exhibited difficulties in all 4 aspects of biological motion perception. However, intercorrelations between test scores were weak in both full-term and preterm children—a finding that supports the view that these processes are relatively independent. Preterm children also displayed more autistic-like traits than full-term peers. In preterm (but not full-term) children, these traits were negatively correlated with performance in the task requiring structure-from-motion processing, r(30) = −.36, p < .05), but positively correlated with the ability to extract identity, r(30) = .45, p < .05). These findings extend previous reports of vulnerability in systems involved in processing dynamic cues in preterm children and suggest that a core deficit in social perception/cognition may contribute to the development of the social and behavioral difficulties even in members of this population who are functioning within the normal range intellectually. The results could inform the development of screening, diagnostic, and intervention tools.

Subcellular mapping of dendritic activity in optic flow processing neurons

Dendritic integration is a fundamental element of neuronal information processing. So far, few studies have provided a detailed spatial picture of this process, describing the properties of local dendritic activity and its subcellular organization. Here, we used 2-photon calcium imaging in optic flow processing neurons of the fly Calliphora vicina to determine the preferred location and direction of local motion cues for small branchlets throughout the entire dendrite. We found a pronounced retinotopic mapping on both the subcellular and the cell population level. In addition, dendritic branchlets residing in different layers of the neuropil were tuned to distinct directions of motion. Summing the local receptive fields of all dendritic branchlets reproduced the characteristic properties of these neurons' axonal output receptive fields. Our results corroborate the notion that the dendritic morphology of vertical system cells allows them to selectively collect local motion inputs with particular directional preferences from a spatially organized input repertoire, thus forming filters that match global patterns of optic flow. Furthermore, we suggest that the facet arrangement across the fly's eye shapes the subcellular direction tuning to local motion stimuli. These data illustrate a highly structured circuit organization as an efficient way to hard-wire a complex sensory task.

Auditory Clicks Distort Perceived Velocity but Only When the System has to Rely on Extraretinal Signals

Previous work has found that repetitive auditory stimulation (click trains) increases the subjective velocity of subsequently presented moving stimuli. We ask whether the effect of click trains is stronger for retinal velocity signals (produced when the target moves across the retina) or for extraretinal velocity signals (produced during smooth pursuit eye movements, when target motion across the retina is limited). In Experiment 1, participants viewed leftward or rightward moving single dot targets, travelling at speeds from 7.5 to 17.5 deg/s. They estimated velocity at the end of each trial. Prior presentation of auditory click trains increased estimated velocity, but only in the pursuit condition, where estimates were based on extraretinal velocity signals. Experiment 2 generalized this result to vertical motion. Experiment 3 found that the effect of clicks during pursuit disappeared when participants tracked across a visually textured background that provided strong local motion cues. Together these results suggest that auditory click trains selectively affect extraretinal velocity signals. This novel finding suggests that the cross-modal integration required for auditory click trains to influence subjective velocity operates at later stages of processing.

The feet have it: Local biological motion cues trigger reflexive attentional orienting in the brain

Most vertebrates, humans included, have a primitive visual system extremely sensitive to the motion of biological entities. Most previous studies have examined the global aspects of biological motion perception, but local motion processing has received much less attention. Here we provide direct psychophysical and electrophysiological evidence that human observers are intrinsically tuned to the characteristics of local biological motion cues independent of global configuration. Using a modified central cueing paradigm, we show that observers involuntarily orient their attention towards the walking direction of feet motion sequences, which triggers an early directing attention negativity (EDAN) in the occipito-parietal region 100–160ms after the stimulus onset. Notably, such effects are sensitive to the orientation of the local cues and are independent of whether the observers are aware of the biological nature of the motion. Our findings unambiguously demonstrate the automatic processing of local biological motion without explicit recognition. More importantly, with the discovery that local biological motion signals modulate attention, we highlight the functional importance of such processing in the brain.

Individual differences in the perception of biological motion: Links to social cognition and motor imagery

Biological motion perception is often claimed to support social cognition, and to rely upon embodied representations and motor imagery. Are people with higher levels of social traits or more vivid motor imagery better at biological motion perception? We administered four experiments measuring sensitivity in using (global) form and (local) motion cues in biological motion, plus well-established measures of social cognition (e.g., empathy) and motor imagery (e.g., kinesthetic motor imagery). This first systematic investigation of individual variability in biological motion processing demonstrated significant relationships between these domains, along with a dissociation. Sensitivity for using form cues in biological motion processing was correlated with social (and not the imagery) measures; sensitivity for using motion cues was correlated with motor imagery (and not the social) measures. These results could not be explained by performance on non-biological control stimuli. We thus show that although both social cognition and motor imagery predict sensitivity to biological motion, these skills likely tap into different aspects of perception.

What do you mean with “direction”? Local and global cues to biological motion perception in pigeons

Biological motion point-light displays are a rich and versatile instrument to study perceptual organization. Humans are able to retrieve information from biological motion through at least two different channels: The global articulated structure as revealed by the non-rigid, yet highly constrained deformation of the dot pattern, and the characteristics of local motion trajectories of individual dots. Here, we tested eight pigeons on a task in which they had to discriminate a left-facing from a right-facing biological motion point-light figure. Since the two stimuli were mirror-flipped versions of each other, we were not sure if the birds would be able to solve the task at all. However, all birds learned the discrimination quickly and performed at high accuracy. We then challenged them with a number of test trials introduced into the sequence of the normal training trials. Tested on backwards moving walkers, the majority of the birds indicated that they used local motion cues to solve the training task, while the remaining birds obviously used global, configural cues. Testing the pigeons on different versions of scrambled biological motion confirmed that each individual bird had made a clear decision for one of the two potentially available strategies. While we confirm a previously described local precedence in processing visual patterns, the fact that some birds used global features suggests that even the birds who relied on local cues probably dispose of the perceptual abilities to use global structure, but “chose” to not use them.

Efficiency of extracting stereo-driven object motions

Most living things and many nonliving things deform as they move, requiring observers to separate object motions from object deformations. When the object is partially occluded, the task becomes more difficult because it is not possible to use two-dimensional (2-D) contour correlations (Cohen, Jain, & Zaidi, 2010). That leaves dynamic depth matching across the unoccluded views as the main possibility. We examined the role of stereo cues in extracting motion of partially occluded and deforming three-dimensional (3-D) objects, simulated by disk-shaped random-dot stereograms set at randomly assigned depths and placed uniformly around a circle. The stereo-disparities of the disks were temporally oscillated to simulate clockwise or counterclockwise rotation of the global shape. To dynamically deform the global shape, random disparity perturbation was added to each disk's depth on each stimulus frame. At low perturbation, observers reported rotation directions consistent with the global shape, even against local motion cues, but performance deteriorated at high perturbation. Using 3-D global shape correlations, we formulated an optimal Bayesian discriminator for rotation direction. Based on rotation discrimination thresholds, human observers were 75% as efficient as the optimal model, demonstrating that global shapes derived from stereo cues facilitate inferences of object motions. To complement reports of stereo and motion integration in extrastriate cortex, our results suggest the possibilities that disparity selectivity and feature tracking are linked, or that global motion selective neurons can be driven purely from disparity cues.

Long trajectory for the development of sensitivity to global and biological motion

We used a staircase procedure to test sensitivity to (1) global motion in random-dot kinematograms moving at 4° and 18° s(-1) and (2) biological motion. Thresholds were defined as (1) the minimum percentage of signal dots (i.e. the maximum percentage of noise dots) necessary for accurate discrimination of upward versus downward motion or (2) the maximum percentage of noise dots tolerated for accurate discrimination of biological from non-biological motion. Subjects were adults and children aged 6-8, 9-11, and 12-14 years (n = 20 per group). Contrary to earlier research, results revealed a similar, long developmental trajectory for sensitivity to global motion at both slower and faster speeds and for biological motion. Thresholds for all three tasks improved monotonically between 6 and 14 years of age, at which point they were adult-like. The results suggest that the extrastriate mechanisms that integrate local motion cues over time and space take many years to mature.

Form and motion make independent contributions to the response to biological motion in occipitotemporal cortex

Psychophysical and computational studies have provided evidence that both form and motion cues are used in the perception of biological motion. However, neuroimaging and neurophysiological studies have suggested that the neural processing of actions in temporal cortex might rely on form cues alone. Here we examined the contribution of form and motion to the spatial pattern of response to biological motion in ventral and lateral occipitotemporal cortex, using functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA). We found that selectivity to intact versus scrambled biological motion in lateral occipitotemporal cortex was correlated with selectivity for bodies and not for motion. However, this appeared to be due to the fact that subtracting scrambled from intact biological motion removes any contribution of local motion cues. Instead, we found that form and motion made independent contributions to the spatial pattern of responses to biological motion in lateral occipitotemporal regions MT, MST, and the extrastriate body area. The motion contribution was position-dependent, and consistent with the representation of contra- and ipsilateral visual fields in MT and MST. In contrast, only form contributed to the response to biological motion in the fusiform body area, with a bias towards central versus peripheral presentation. These results indicate that the pattern of response to biological motion in ventral and lateral occipitotemporal cortex reflects the linear combination of responses to form and motion.