Motion Extrapolation Research Articles

Visual motion extrapolation of moving objects drives real-time temporal re-alignment across hierarchical neural position representations

Visual motion extrapolation of moving objects drives real-time temporal re-alignment across hierarchical neural position representations

Repulsive Shells

This paper develops a shape space framework for collision-aware geometric modeling, where basic geometric operations automatically avoid inter-penetration. Shape spaces are a powerful tool for surface modeling, shape analysis, nonrigid motion planning, and animation, but past formulations permit nonphysical intersections. Our framework augments an existing shape space using a repulsive energy such that collision avoidance becomes a first-class property, encoded in the Riemannian metric itself. In turn, tasks like intersection-free shape interpolation or motion extrapolation amount to simply computing geodesic paths via standard numerical algorithms. To make optimization practical, we develop an adaptive collision penalty that prevents mesh self-intersection, and converges to a meaningful limit energy under refinement. The final algorithms apply to any category of shape, and do not require a dataset of examples, training, rigging, nor any other prior information. For instance, to interpolate between two shapes we need only a single pair of meshes with the same connectivity. We evaluate our method on a variety of challenging examples from modeling and animation.

Motion extrapolation in sport expertise: Representational momentum and representational gravity in volleyball athletes

When people indicate the vanishing location of a moving target that suddenly disappears, systematic errors forward (in the direction of motion) and downward (in the direction of gravity) emerge. These spatial displacements were coined, respectively, Representational Momentum and Representational Gravity, and are believed to reflect internalized ecologically relevant physical invariants useful for the anticipation of future states of an event. Previous research has shown that sports athletes exhibit increased Representational Momentum, indicating enhanced motion extrapolation and anticipation, albeit it is still not clear up to what degree this effect is specific for the expertise context or if it generalizes to other dynamic events. Furthermore, the influence of expertise on Representational Gravity, particularly in contexts where anticipation of vertically moving objects is crucial, remains understudied. This study aimed to address these gaps by focusing on Volleyball as a context of expertise due to the prevalence of fast vertically moving balls. Volleyball athletes and non-athletes indicated the perceived offset location of a smoothly moving target, which moved at a constant speed or was subjected to acceleration/deceleration, embedded either in a Volleyball or neutral context. Outcomes revealed that for the Volleyball context, athletes, but not non-athletes, revealed a significant trend to misperceive targets moving along the left diagonal to be further displaced forward beyond what would be expected due to Representational Momentum alone. This finding is discussed in relation to the natural statistics of Volleyball games, where crossed ball trajectories, particularly by the outside hitter, are more prevalent, fast, and offensive, requiring better anticipation to be efficiently dealt with.

Corrective mechanisms of motion extrapolation.

Transmission and processing of sensory information in the visual system takes time. For motion perception, our brain can overcome this intrinsic neural delay through extrapolation mechanisms and accurately predict the current position of a continuously moving object. But how does the system behave when the motion abruptly changes and the prediction becomes wrong? Here we address this question by studying the perceived position of a moving object with various abrupt motion changes by human observers. We developed a task in which a bar is monotonously moving horizontally, and then motion suddenly stops, reverses, or disappears-then-reverses around two vertical stationary reference lines. Our results showed that participants overestimated the position of the stopping bar but did not perceive an overshoot in the motion reversal condition. When a temporal gap was added at the reverse point, the perceptual overshoot of the end point scaled with the gap durations. Our model suggests that the overestimation of the object position when it disappears is not linear as a function of its speeds but gradually fades out. These results can thus be reconciled in a single process where there is an interplay of the cortical motion prediction mechanisms and the late sensory transient visual inputs.

The impact of visually simulated self-motion on predicting object motion.

To interact successfully with moving objects in our environment we need to be able to predict their behavior. Predicting the position of a moving object requires an estimate of its velocity. When flow parsing during self-motion is incomplete-that is, when some of the retinal motion created by self-motion is incorrectly attributed to object motion-object velocity estimates become biased. Further, the process of flow parsing should add noise and lead to object velocity judgements being more variable during self-motion. Biases and lowered precision in velocity estimation should then translate to biases and lowered precision in motion extrapolation. We investigated this relationship between self-motion, velocity estimation and motion extrapolation with two tasks performed in a realistic virtual reality (VR) environment: first, participants were shown a ball moving laterally which disappeared after a certain time. They then indicated by button press when they thought the ball would have hit a target rectangle positioned in the environment. While the ball was visible, participants sometimes experienced simultaneous visual lateral self-motion in either the same or in the opposite direction of the ball. The second task was a two-interval forced choice task in which participants judged which of two motions was faster: in one interval they saw the same ball they observed in the first task while in the other they saw a ball cloud whose speed was controlled by a PEST staircase. While observing the single ball, they were again moved visually either in the same or opposite direction as the ball or they remained static. We found the expected biases in estimated time-to-contact, while for the speed estimation task, this was only the case when the ball and observer were moving in opposite directions. Our hypotheses regarding precision were largely unsupported by the data. Overall, we draw several conclusions from this experiment: first, incomplete flow parsing can affect motion prediction. Further, it suggests that time-to-contact estimation and speed judgements are determined by partially different mechanisms. Finally, and perhaps most strikingly, there appear to be certain compensatory mechanisms at play that allow for much higher-than-expected precision when observers are experiencing self-motion-even when self-motion is simulated only visually.

Does a concurrent motor process influence representational momentum?

Whether the direction of a hand motion that is congruent or incongruent with a concurrent target motion can influence representational momentum for that target was examined. Participants viewed a leftward or rightward moving target while moving their hand rightward, leftward, or not moving their hand. Prior studies of mental rotation found that congruency or incongruency of the direction of mental rotation and the direction of a concurrent physical rotation of a stimulus influenced mental rotation. As mental rotation and representational momentum each involve extrapolation of target motion, it could be predicted that congruency of the direction of hand motion and the direction of target motion might influence representational momentum of the target. Robust representational momentum occurred in all conditions, but there was no effect of congruency of hand motion and target motion, nor of the presence or absence of hand motion, on representational momentum. The results are consistent with a hypothesis that the generation of representational momentum involves sensory processes rather than motor processes.

Spike-timing dependent plasticity partially compensates for neural delays in a multi-layered network of motion-sensitive neurons.

The ability of the brain to represent the external world in real-time is impacted by the fact that neural processing takes time. Because neural delays accumulate as information progresses through the visual system, representations encoded at each hierarchical level are based upon input that is progressively outdated with respect to the external world. This 'representational lag' is particularly relevant to the task of localizing a moving object-because the object's location changes with time, neural representations of its location potentially lag behind its true location. Converging evidence suggests that the brain has evolved mechanisms that allow it to compensate for its inherent delays by extrapolating the position of moving objects along their trajectory. We have previously shown how spike-timing dependent plasticity (STDP) can achieve motion extrapolation in a two-layer, feedforward network of velocity-tuned neurons, by shifting the receptive fields of second layer neurons in the opposite direction to a moving stimulus. The current study extends this work by implementing two important changes to the network to bring it more into line with biology: we expanded the network to multiple layers to reflect the depth of the visual hierarchy, and we implemented more realistic synaptic time-courses. We investigate the accumulation of STDP-driven receptive field shifts across several layers, observing a velocity-dependent reduction in representational lag. These results highlight the role of STDP, operating purely along the feedforward pathway, as a developmental strategy for delay compensation.

An Integrated Model of Collaborative Skill Acquisition: Anticipation, Control Tuning, and Role Adoption.

We studied collaborative skill acquisition in a dynamic setting with the game Co-op Space Fortress. While gaining expertise, the majority of subjects became increasingly consistent in the role they adopted without being able to communicate. Moreover, they acted in anticipation of the future task state. We constructed a collaborative skill acquisition model in the cognitive architecture ACT-R that reproduced subject skill acquisition trajectory. It modeled role adoption through reinforcement learning and predictive processes through motion extrapolation and learned relevant control parameters using both a reinforcement learning procedure and a new to ACT-R supervised learning procedure. This is the first integrated cognitive model of collaborative skill acquisition and, as such, gives us valuable insights into the multiple cognitive processes that are involved in learning tocollaborate.

Long time extrapolation of DEM with heat conduction in a moving granular medium

This paper presents a novel approach for extrapolating DEM simulations of heat transfer over a long period of time. This method is an extension of a previously published algorithm for granular motion extrapolation, introducing heat transfer. The main idea is to perform a short-term DEM simulation for one period and then apply a conductive heat transfer extrapolation algorithm. This strategy is tested over a pilot-scale rotating drum. The outcomes of standard and extrapolated DEM simulations are compared. The results are very similar while the computational time is reduced by a factor greater than 100.

The lateralized flash-lag illusion: A psychophysical and pupillometry study

The flash-lag illusion (FLI) is a visual phenomenon where a flashed object, either co-localized or in physical alignment with another continuously moving object, is perceived to lag behind the path of the moving object. In the present study, we reveal an anisotropy of the FLI between the lateral visual fields that was expressed psychophysically as different points of subjective equality, depending on the hemifield in which the stimuli appeared. Specifically, the study confirmed that, as seen in two previous studies, the FLI was significantly larger in the left visual field (LVF) than in the right (RVF). In addition, pupil dilations were larger in the RVF than in the LVF as well as returning to baseline levels more rapidly in the LVF. We interpret these findings as converging on revealing more efficient spatial and attentional processing and, in turn, extrapolation of motion in the LVF/right hemisphere than in the RVF/left hemisphere.

Monocular Quasi-Dense 3D Object Tracking.

A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform. The object association leverages quasi-dense similarity learning to identify objects in various poses and viewpoints with appearance cues only. After initial 2D association, we further utilize 3D bounding boxes depth-ordering heuristics for robust instance association and motion-based 3D trajectory prediction for re-identification of occluded vehicles. In the end, an LSTM-based object velocity learning module aggregates the long-term trajectory information for more accurate motion extrapolation. Experiments on our proposed simulation data and real-world benchmarks, including KITTI, nuScenes, and Waymo datasets, show that our tracking framework offers robust object association and tracking on urban-driving scenarios. On the Waymo Open benchmark, we establish the first camera-only baseline in the 3D tracking and 3D detection challenges. Our quasi-dense 3D tracking pipeline achieves impressive improvements on the nuScenes 3D tracking benchmark with near five times tracking accuracy of the best vision-only submission among all published methods.

The impact of visually simulated self-motion on predicting object motion-A registered report protocol.

To interact successfully with moving objects in our environment we need to be able to predict their behavior. Predicting the position of a moving object requires an estimate of its velocity. When flow parsing during self-motion is incomplete-that is, when some of the retinal motion created by self-motion is incorrectly attributed to object motion-object velocity estimates become biased. Further, the process of flow parsing should add noise and lead to object velocity judgements being more variable during self-motion. Biases and lowered precision in velocity estimation should then translate to biases and lowered precision in motion extrapolation. We investigate this relationship between self-motion, velocity estimation and motion extrapolation with two tasks performed in a realistic virtual reality (VR) environment: first, participants are shown a ball moving laterally which disappears after a certain time. They then indicate by button press when they think the ball would have hit a target rectangle positioned in the environment. While the ball is visible, participants sometimes experience simultaneous visual lateral self-motion in either the same or in the opposite direction of the ball. The second task is a two-interval forced choice task in which participants judge which of two motions is faster: in one interval they see the same ball they observed in the first task while in the other they see a ball cloud whose speed is controlled by a PEST staircase. While observing the single ball, they are again moved visually either in the same or opposite direction as the ball or they remain static. We expect participants to overestimate the speed of a ball that moves opposite to their simulated self-motion (speed estimation task), which should then lead them to underestimate the time it takes the ball to reach the target rectangle (prediction task). Seeing the ball during visually simulated self-motion should increase variability in both tasks. We expect to find performance in both tasks to be correlated, both in accuracy and precision.

Seeing and extrapolating motion trajectories share common informative activation patterns in primary visual cortex.

The natural environment is dynamic and moving objects become constantly occluded, engaging the brain in a challenging completion process to estimate where and when the object might reappear. Although motion extrapolation is critical in daily life-imagine crossing the street while an approaching car is occluded by a larger standing vehicle-its neural underpinnings are still not well understood. While the engagement of low-level visual cortex during dynamic occlusion has been postulated, most of the previous group-level fMRI-studies failed to find evidence for an involvement of low-level visual areas during occlusion. In this fMRI-study, we therefore used individually defined retinotopic maps and multivariate pattern analysis to characterize the neural basis of visible and occluded changes in motion direction in humans. To this end, participants learned velocity-direction change pairings (slow motion-upwards; fast motion-downwards or vice versa) during a training phase without occlusion and judged the change in stimulus direction, based on its velocity, during a following test phase with occlusion. We find that occluded motion direction can be predicted from the activity patterns during visible motion within low-level visual areas, supporting the notion of a mental representation of motion trajectory in these regions during occlusion.

Interocular Transfer: The Dichoptic Flash-Lag Effect in Controls and Amblyopes.

PurposeThe mammalian brain can take into account the neural delays in visual information transmission from the retina to the cortex when accurately localizing the instantaneous position of moving objects by motion extrapolation. In this study, we wanted to investigate whether such extrapolation mechanism operates in a comparable fashion between the eyes in normally sighted and amblyopic observers.MethodsTo measure interocular extrapolation, we adapted a dichoptic version of the flash-lag effect (FLE) paradigm, in which a flashed bar is perceived to lag behind a moving bar when their two positions are physically aligned. Twelve adult subjects with amblyopia and 12 healthy controls participated in the experiment. We measured the FLE magnitude of the subjects under binocular, monocular, and dichoptic conditions.ResultsIn controls, the FLE magnitude of binocular condition was significantly smaller than that of monocular conditions (P ≤ 0.023), but there was no difference between monocular and dichoptic conditions. Subject with amblyopia exhibited a smaller FLE magnitude in the dichoptic condition when the moving bar was presented to the amblyopic eye and the flash to the fellow eye (DA condition) compared to the opposite way around (DF condition), consistent with a delay in the processing of the amblyopic eye (P = 0.041).ConclusionsOur observations confirm that trajectory extrapolation mechanisms transfer between the eyes of normal observers. However, such transfer may be impaired in amblyopia. The smaller FLE magnitude in DA compared to DF in patients with amblyopia could be due to an interocular delay in the amblyopic visual system. The observation that normal controls present a smaller FLE in binocular conditions raises the question whether a larger FLE is or is not an indicator of better motion processing and extrapolation.

Beyond motion extrapolation: vestibular contribution to head-rotation-induced flash-lag effects.

The perceived position of a flash aligned with a moving object usually lags behind that object. This illusion is well known as the flash-lag effect. Interestingly, head rotation alone can also induce a flash-lag effect. To date, the underlying mechanism for the head-rotation-induced flash-lag effect remains unclear. Using a virtual reality approach, we examined the contribution of vestibular signal processing in producing the effect. We found that vestibular, rather than kinesthetic, signal processing is critical for this type of flash-lag effect to occur. When head rotation induced a stationary reference stimulus in space to move on the retina, we observed a flash-lead effect relative to the reference (or a flash-lag effect relative to the head). Moreover, after a short-term adaptation training on a novel association between head rotation and retinal motion, the direction of the flash-lag effect was consistent with the newly trained association. These findings disagree with a previous account extended from the influential motion extrapolation hypothesis. Rather, they support a cross-modal bias hypothesis that the visual-vestibular associations developed from multisensory experiences may generate biasing visual signals in the associated direction with the vestibular signals, which help produce the head-rotation-induced flash-lag effects. Our findings may provide new insight into other multisensory integration phenomena.

A comparison of satellite cloud motion vectors techniques to forecast intra-day hourly solar global horizontal irradiation

Solar forecasting provides valuable information for grid management. Satellite-based forecasting tools account for the short-term intra-day time horizons, typically outperforming numerical weather predictions up to 4–5 h ahead. The method consist of three separated stages, namely, cloud motion estimation, motion extrapolation and satellite-to-irradiation conversion. In this work we compare different satellite-based proposals for hourly irradiation forecast up to 5 h ahead using a 2-years data set. The widely-used Lorenz’s block matching technique and four optical flow (OF) algorithms are assessed, both at image and irradiation levels. All the methods are locally optimized to obtain their peak performance. It is found that the OF algorithm which combines an L1 data penalty term on the optical flow equation with total variation regularization (TVL1) outperforms the rest. Different image extrapolation approaches and spatial smoothing are also tested. It is found that changing the extrapolation technique does not have much impact in the overall performance and that important gains can be obtained by optimally smoothing the predicted images previous to solar irradiation conversion. By doing this, all methods outperform the exigent convex persistence benchmark, achieving positive forecasting skills. The tests are performed using GOES-East satellite images of south-east South America, and the methods’ optimal parameters are given.

Delayed Correction for Extrapolation in Amblyopia.

PurposeIt has been suggested that amblyopes present impaired motion extrapolation mechanisms. In this study, we used the flash grab effect (FGE), the illusory mislocalization of a briefly flashed stimulus in the direction of a reversing moving background, to investigate whether the amblyopic visual system can correct overextrapolation.MethodsThirteen amblyopes and 13 control subjects participated in the experiment. We measured the monocular FGE magnitude for each subject. Two spatial frequency (2 and 8 cycles), two texture configurations (square wave or sine wave), and two speed conditions (270 degrees/s and 67.5 degrees/s) were tested. In addition, control subjects were further tested in reduced luminance conditions.ResultsCompared with controls, amblyopes exhibited a larger FGE magnitude both in their fellow eye (FE) and amblyopic eye (AE). The FGE magnitude of their AE was significantly larger than that of the FE. In a control experiment, we observed that the FGE magnitude increases with the decreasing of the luminance. The FGE magnitude of amblyopes fall into the same range as that of controls under reduced luminance conditions.ConclusionsWe observed a lager FGE in patients with amblyopia, which indicates that the amblyopic visual system does not accurately correct the overextrapolation when a moving object abruptly reverses its direction. This spatiotemporal processing deficit could be ascribed to delayed visual processing in the amblyopic visual system.

Motion extrapolation in the flash-lag effect depends on perceived, rather than physical speed

In the flash-lag effect (FLE), a flash in spatiotemporal alignment with a moving object is misperceived as lagging behind the moving object. One proposed explanation for this illusion is based on predictive motion extrapolation of trajectories. In this interpretation, the diverging effects of velocity on the perceived position of the moving object suggest that FLE might be based on the neural representation of perceived, rather than physical, velocity. By contrast, alternative explanations based on differential latency or temporal averaging would predict that the FLE does not rely on such a representation of perceived velocity. Here we examined whether the FLE is sensitive to illusory changes in perceived speed that result in changes to perceived velocity, while physical speed is constant. The perceived speed of the moving object was manipulated using revolving wedge stimuli with variable pattern textures (Experiment 1) and luminance contrast (Experiment 2). The motion extrapolation interpretation would predict that the changes in FLE magnitude should correspond to the changes in the perceived speed of the moving object. In the current study, two experiments demonstrated that perceived speed and FLE magnitude increased in the dynamic pattern relative to the static pattern conditions, and that the same effect was found in the low contrast compared to the high contrast conditions. These results showed that manipulations of texture and contrast that are known to alter judgments of perceived speed also modulate perceived position. We interpret this as a consequence of motion extrapolation mechanisms and discuss possible explanations for why we observed no cross-effect correlation.

Early and late evoked brain responses differentially reflect feature encoding and perception in the flash-lag illusion

In the flash-lag illusion (FLI), the position of a flash presented ahead of a moving bar is mislocalized, so the flash appears to lag the bar. Currently, it is not clear whether this effect is due to early perceptual-related neural processes such as motion extrapolation or reentrant processing, or due to later feedback processing relating to postdiction, i.e., retroactively altered perception. We presented 17 participants with the FLI paradigm while recording EEG. A central flash occurred either 51 ms (“early”) or 16 ms (“late”) before the bar moving from left to right reached the screen center. Participants judged whether the flash appeared to the right (“no flash lag illusion”) or to the left (“flash-lag illusion”) of the bar. Using single-trial linear modeling, we examined the influence of timing (“early” vs. “late”) and perception (“illusion” vs. “no illusion”) on flash-evoked brain responses and estimated the cortical sources underlying the FLI. An earlier frontal and occipital component (200–276 ms) differentiated time-locked early vs. late stimulus presentation, indicating that early evoked brain responses reflect feature encoding in the FLI. Perception of the FLI was associated with a late window (368–452 ms) in the ERP, with larger deflections for illusion than no illusion trials, localized to the left inferior occipital gyrus. This suggests a postdiction-related reconstruction of ambiguous sensory stimulation involving late processes in the occipito-temporal cortex, previously associated with temporal integration phenomena. Our findings indicate that perception of the FLI relies on an interplay between ongoing stimulus encoding of the moving bar and feedback processing of the flash, which takes place at later integration stages.

Motion behind occluder: Amodal perception and visual motion extrapolation

ABSTRACT One of the main challenges that our visual system must resolve is to judge the current position of an occluded moving object. This computation is known as motion extrapolation (ME), and it is required to perform several tasks: for example, judging when an occluded moving target collides with a given cue or whether it reappeared in time. These tasks may easily be encountered in more ecological settings, highlighting the importance of ME in our daily life. The goal of the current review is to synthesize the existing literature that investigated targets during occluded motion, in order to provide a general overview of the topic. To this aim, different aspects are discussed: occluded motion as a true perceptual phenomenon, the commonly used tasks in the literature, how perceptual and cognitive factors may affect individual performance, which mechanisms are supposed to be involved during ME and, finally, the neural correlates of occluded motion perception.