Motion Cues Research Articles

Monkeys engage in visual simulation to solve complex problems.

Visual simulation-i.e., using internal reconstructions of the world to experience potential future versions of events that are not currently happening-is among the most sophisticated capacities of the human mind. But is this ability in fact uniquely human? To answer this question, we tested monkeys on a series of experiments involving the "Planko" game, which we have previously used to evoke visual simulation in human participants. We found that monkeys were able to successfully play the game using a simulation strategy, predicting the trajectory of a ball through a field of planks while demonstrating a level of accuracy and behavioral signatures comparable with those of humans. Computational analyses further revealed that the monkeys' strategy while playing Planko aligned with a recurrent neural network (RNN) that approached the task using a spontaneously learned simulation strategy. Finally, we carried out awake functional magnetic resonance imaging while monkeys played Planko. We found activity in motion-sensitive regions of the monkey brain during hypothesized simulation periods, even without any perceived visual motion cues. This neural result closely mirrors previous findings from human research, suggesting a shared mechanism of visual simulation across species. Taken together, these findings challenge traditional views of animal cognition, proposing that nonhuman primates possess a complex cognitive landscape, capable of invoking imaginative and predictive mental experiences to solve complex everyday problems.

Counterillumination reduces bites by Great White sharks.

In the open ocean, achieving camouflage is complicated by the fact that the downwelling light is generally much brighter than the upwelling light, which means that any object, even if its ventral surface is white due to countershading, will appear as a dark silhouette when viewed from below.1,2,3 To overcome this, many marine species employ counterillumination, whereby light is emitted from photophores on their ventral surface to replace the downwelling light blocked by their body.4,5,6 However, only a single behavioral study has tested the efficacy of counterillumination as an anti-predation strategy.7 Counterillumination is predicted to be particularly useful against predators that have poor visual acuity and lack color vision,8 like the Great White shark (Carcharodon carcharias), the species responsible for most human shark-bite fatalities globally.9 Here, we take inspiration from nature to show that counterillumination can prevent Great White sharks from attacking artificial seal decoys. Using seal decoys fitted with LED lights and towed behind a boat, we explored the efficiency of different light configurations on the deterrence effect, showing that visual shape and motion cues are critical for prey recognition by Great White sharks. Counterillumination that is brighter than the background is most effective in deterring sharks, implying that, in this context, counterillumination works through disruptive camouflage rather than background matching. Our results reveal the importance of a dark silhouette against a lighter background in predatory behavior in Great White sharks and that altering the silhouette may form the basis of new non-invasive shark deterrent technology to protect human life. VIDEO ABSTRACT.

Bilateral interactions of optic-flow sensitive neurons coordinate course control in flies

Animals rely on compensatory actions to maintain stability and navigate their environment efficiently. These actions depend on global visual motion cues known as optic-flow. While the optomotor response has been the traditional focus for studying optic-flow compensation in insects, its simplicity has been insufficient to determine the role of the intricate optic-flow processing network involved in visual course control. Here, we reveal a series of course control behaviours in Drosophila and link them to specific neural circuits. We show that bilateral electrical coupling of optic-flow-sensitive neurons in the fly’s lobula plate are required for a proper course control. This electrical interaction works alongside chemical synapses within the HS-H2 network to control the dynamics and direction of turning behaviours. Our findings reveal how insects use bilateral motion cues for navigation, assigning a new functional significance to the HS-H2 network and suggesting a previously unknown role for gap junctions in non-linear operations.

Evaluating the Effects of Motion Cues in Virtual Truck Driver Training

The integration of simulation technology into professional training has transformed learning in various sectors. Although prior research has explored motion cueing in simulators such as aviation training, driver navigation and performance, there is limited investigation into the impact of motion stimuli in VR-based truck driver training. This study investigates the driving performance of truck drivers in two virtual driving tasks, comparing conditions with and without motion cues. Participants wore virtual reality headsets and were situated within an immersive motion simulator. They completed two driving tasks involving truck docking and highway driving. Simulator sickness, stress, and user experience were assessed using the simulator sickness questionnaire, subjective unit of distress scale, and user experience questionnaire, respectively. Our findings suggest that motion cues hold potential to enhance subjective experiences, making driving more engaging, stimulating, and mitigating motion sickness. However, they did not significantly affect overall driving performance, except for steering behavior. Participants exhibited more engaged steering during highway driving when motion cues were present. Our findings suggest that the impact of motion cues is task-dependent, with their absence not showing a significant detrimental effect on driving performance in the simulated environment. Further validation with field-observed data is necessary to confirm these findings. These insights can guide future policies and regulations governing virtual truck driver training, ultimately ensuring improved safety and performance in the industry.

Road-crossing behavior and safety of pedestrians facing autonomous vehicles with an acceleration indicator eHMI in VR traffic flow

To support pedestrian road-crossing behavior in the presence of autonomous vehicles (AVs), we propose a novel external Human-Machine Interface (eHMI) that enhances the vehicle’s inherent motion cues, which pedestrians already use in traditional traffic. Based on the Beta Movement, the eHMI design consists of light bands with multiple arrows moving forward or backward to indicate the vehicle’s accelerating or decelerating motion state. We measured pedestrians’ behavior and perceived safety when faced with AVs equipped with this acceleration indicator eHMI in a virtual reality (VR) street scene, where pedestrians had road-crossing tasks and vehicles did not always yield. We found that pedestrians exhibited crossing behavior according to the motion state of the oncoming vehicle. They almost always crossed if the vehicle yielded but considered the gap size to cross when it did not. With the acceleration indicator eHMI, they crossed more frequently if the vehicle yielded, yet less frequently if it did not. Although risky crossings could not be fully prevented, pedestrians crossed faster and maintained larger safety margins when the non-yielding vehicle used this eHMI. Additionally, pedestrians started to cross earlier when the eHMI indicated the vehicle’s decelerating state. Regarding pedestrians’ perceived safety, the eHMI did not increase perceived safety for decelerating vehicles but effectively decreased the perceived safety for accelerating vehicles. These results demonstrate the potential of enhancing vehicles’ inherent motion cues with eHMIs, but also imply that making vehicles’ motion state more apparent might not simply increase pedestrians’ perceived safety. Overall, a trade-off seems to exist that it is pedestrians’ sense of unsafety that leads to their safe road-crossing behavior. Suggestions for future research and VR experiments were also discussed.

Visual mental imagery of nonpredictive central social cues triggers automatic attentional orienting

Previous research has demonstrated that social cues (e.g., eye gaze, walking direction of biological motion) can automatically guide people's focus of attention, a well-known phenomenon called social attention. The current research shows that voluntarily generated social cues via visual mental imagery, without being physically presented, can produce robust attentional orienting similar to the classic social attentional orienting effect. Combining a visual imagery task with a dot-probe task, we found that imagining a non-predictive gaze cue could orient attention towards the gazed-at hemifield. Such attentional effect persisted even when the imagery gaze cue was counter-predictive of the target hemifield, and could be generalized to biological motion cue. Besides, this effect could not be simply attributed to low-level motion signal embedded in gaze cues. More importantly, an eye-tracking experiment carefully monitoring potential eye movements demonstrated the imagery-induced attentional orienting effect induced by social cues, but not by non-social cues (i.e., arrows), suggesting that such effect is specialized to visual imagery of social cues. These findings accentuate the demarcation between social and non-social attentional orienting, and may take a preliminary step in conceptualizing voluntary visual imagery as a form of internally directed attention.

Visual Perceptual Learning of Form-Motion Integration: Exploring the Involved Mechanisms with Transfer Effects and the Equivalent Noise Approach.

Background: Visual perceptual learning plays a crucial role in shaping our understanding of how the human brain integrates visual cues to construct coherent perceptual experiences. The visual system is continually challenged to integrate a multitude of visual cues, including form and motion, to create a unified representation of the surrounding visual scene. This process involves both the processing of local signals and their integration into a coherent global percept. Over the past several decades, researchers have explored the mechanisms underlying this integration, focusing on concepts such as internal noise and sampling efficiency, which pertain to local and global processing, respectively. Objectives and Methods: In this study, we investigated the influence of visual perceptual learning on non-directional motion processing using dynamic Glass patterns (GPs) and modified Random-Dot Kinematograms (mRDKs). We also explored the mechanisms of learning transfer to different stimuli and tasks. Specifically, we aimed to assess whether visual perceptual learning based on illusory directional motion, triggered by form and motion cues (dynamic GPs), transfers to stimuli that elicit comparable illusory motion, such as mRDKs. Additionally, we examined whether training on form and motion coherence thresholds improves internal noise filtering and sampling efficiency. Results: Our results revealed significant learning effects on the trained task, enhancing the perception of dynamic GPs. Furthermore, there was a substantial learning transfer to the non-trained stimulus (mRDKs) and partial transfer to a different task. The data also showed differences in coherence thresholds between dynamic GPs and mRDKs, with GPs showing lower coherence thresholds than mRDKs. Finally, an interaction between visual stimulus type and session for sampling efficiency revealed that the effect of training session on participants' performance varied depending on the type of visual stimulus, with dynamic GPs being influenced differently than mRDKs. Conclusion: These findings highlight the complexity of perceptual learning and suggest that the transfer of learning effects may be influenced by the specific characteristics of both the training stimuli and tasks, providing valuable insights for future research in visual processing.

Dazzling damselfish: investigating motion dazzle as a defence strategy in humbug damselfish (Dascyllus aruanus)

Many animals possess high-contrast body patterns. When moving, these patterns may create confusing or conflicting visual cues that affect a predator’s ability to visually target or capture them, a phenomenon called motion dazzle. The dazzle patterns may generate different forms of optical illusion that can mislead observers about the shape, speed, trajectory and range of the animal. Moreover, it is possible that the disruptive visual effects of the high contrast body patterns can be enhanced when moving against a high contrast background. In this study, we used the humbug damselfish (Dascyllus aruanus) to model the apparent motion cues of its high contrast body stripes against high contrast background gratings of different widths and orientations, from the perspective of a predator. We found with higher frequency gratings, when the background is indiscriminable to a viewer, that the humbugs may rely on the confusing motion cues created by internal stripes. With lower frequency gratings, where the background is likely perceivable by a viewer, the humbugs can rely more on confusing motion cues induced by disruption of edges from both the background and body patterning. We also assessed whether humbugs altered their behaviour in response to different backgrounds. Humbugs remained closer and moved less overall in response to backgrounds with a spatial structure similar to their own striped body pattern, possibly to stay camouflaged against the background and thus avoid revealing themselves to potential predators. At backgrounds with higher frequency gratings, humbugs moved more which may represent a greater reliance on the internal contrast of the fish’s striped body pattern to generate motion dazzle. It is possible that the humbug stripes provide multiple protective strategies depending on the context and that the fish may alter their behaviour depending on the background to maximise their protection.

Saliency detection for underwater moving object with sonar based on motion estimation and multi-trajectory analysis

Due to the limited detection range of optical sensors in the ocean, sonar stands out as the predominant method for detecting dynamic objects in the deep sea. Since there is stronger reflection underwater, the objects are more salient in sonar images compared to the background. In this paper, we propose a saliency detection framework for underwater moving object, which consists of three stages. In the first stage, we use optical flow to get rough global motion cues under background interference in unstable sonar platforms. In the second stage, we propose a trajectory analysis paradigm, which converts the motion cues into isolated connected domains for tracking, and then evaluates the trajectories to get the path of real target. In the third stage, we propose a salient object detection (SOD) model that predicts local saliency maps through feature fusion and multi-level supervision. The global saliency map is then obtained by remapping. Finally, extensive experiments conducted on eight sonar videos demonstrate that our proposed methodology outperforms fifteen other saliency object detection approaches.

The role of binocular disparity and attention in the neural representation of multiple moving stimuli in the visual cortex.

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual process, with stereoscopic depth and motion serving as crucial cues. However, how the visual system uses these cues to segment multiple objects is not fully understood. We investigated how neurons in the middle-temporal (MT) cortex of macaque monkeys represent overlapping surfaces at different depths, moving in different directions. Neuronal activity was recorded from three male monkeys during discrimination tasks under varying attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the binocular disparity of one surface over the other. The disparity bias of a neuron was positively correlated with the neuron's disparity preference for a single surface. In two animals, neurons preferring near disparities of single surfaces (near neurons) showed a near bias for overlapping stimuli, while neurons preferring far disparities (far neurons) showed a far bias. In the third animal, both near and far neurons displayed a near bias, though the near neurons showed a stronger near bias. All three animals exhibited an initial near bias across neurons relative to the average of the responses to the individual surfaces. Although attention modulated neuronal responses, the disparity bias was not caused by attention. We also found that the effect of attention was consistent with object-based, rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. This model is a novel extension of the standard normalization model and provides a unified explanation for the disparity bias across animals. Our results reveal how MT neurons encode multiple stimuli moving at different depths and present new evidence of response modulation by object-based attention. The disparity bias allows subgroups of neurons to preferentially represent individual surfaces of multiple stimuli at different depths, thereby facilitating segmentation.

Problem of Low-Frequency Motion Cueing Along Roll on Flight Simulators

An innovative approach to improve the fidelity of low-frequency lateral motion cueing within full-flight simulators for non-maneuvering aircraft is presented. On the basis of the perception peculiarities by the human vestibular system of movement along the roll, the problem of low-frequency motion cueing of lateral movement by motion systems of flight simulators was formulated and solved. The critical challenge of simulating the set of angular motion cues that pilots perceive during flights is decided. This research highlights two key outcomes. The fourth-order low-pass filter effectively extracts low-frequency motion cues from aircraft motion kinematic parameters, enhancing simulation accuracy. Secondly, proposed method significantly expands the range of simulated motion cues while ensuring their synchronization with high-frequency motion cues along relevant degrees of freedom. This formulation of the problem increases the range of simulated motion cues to ±0,3nz, which practically corresponds to the range of simulated motion cues of a transport aircraft, and thus increases the quality of motion cueing. The implementation of the developed method on the An-72TK-200 full flight simulator confirmed its effectiveness. In conclusion, this study introduces a promising methodology that enhances the quality of motion cueing, thus rendering flight simulations more realistic and beneficial for both pilots and aerospace engineering researchers in the real of non-maneuvering aircraft flight simulators.

Combining surface reflectance and motion cues in peripheral target detection

Combining surface reflectance and motion cues in peripheral target detection

Rats rely on airflow cues for self-motion perception

Self-motion perception is a vital skill for all species. It is an inherently multisensory process that combines inertial (body-based) and relative (with respect to the environment) motion cues. Although extensively studied in human and non-human primates, there is currently no paradigm to test self-motion perception in rodents using both inertial and relative self-motion cues. We developed a novel rodent motion simulator using two synchronized robotic arms to generate inertial, relative, or combined (inertial and relative) cues of self-motion. Eight rats were trained to perform a task of heading discrimination, similar to the popular primate paradigm. Strikingly, the rats relied heavily on airflow for relative self-motion perception, with little contribution from the (limited) optic flow cues provided-performance in the dark was almost as good. Relative self-motion (airflow) was perceived with greater reliability vs. inertial. Disrupting airflow, using a fan or windshield, damaged relative, but not inertial, self-motion perception. However, whiskers were not needed for this function. Lastly, the rats integrated relative and inertial self-motion cues in a reliability-based (Bayesian-like) manner. These results implicate airflow as an important cue for self-motion perception in rats and provide a new domain to investigate the neural bases of self-motion perception and multisensory processing in awake behaving rodents.

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

The influence of form on motion signal processing in the ventral intraparietal area of macaque monkeys

The visual system relies on both motion and form signals to perceive the direction of self-motion, yet the coordination mechanisms between these two elements in this process remain elusive. In the current study, we employed heading perception as a model to delve into the interaction characteristics between form and motion signals. We recorded the responses of neurons in the ventral intraparietal area (VIP), an area with strong heading selectivity, to motion-only, form-only, and combined stimuli of simulated self-motion. Intriguingly, VIP neurons responded to form-only cues defined by Glass patterns, although they exhibited no tuning selectivity. In combined condition, introducing a small offset between form and motion cues significantly enhanced neuronal sensitivity to motion cues. However, with a larger offset, the enhancement effect on sensitivity became comparatively smaller. Moreover, we observed that the influence of form cues on neuronal response to motion cues is more pronounced in the later stage (1–2 s) of stimulation, with a relatively smaller effect in the early stage (0–1 s). This suggests a dynamic interaction between motion and form cues over time for heading perception. In summary, our study uncovered that in area VIP, form information plays a role in constructing accurate self-motion perception. This adds valuable insights into the complex dynamics of how the brain integrates motion and form cues for the perception of one's own movements.

A review of carsickness mitigation: Navigating challenges and exploiting opportunities in the era of intelligent vehicles

Motion sickness (MS) has long been a common complaint in road transportation. However, in the era of driving automation, MS has become an increasingly significant issue. The future intelligent vehicle is envisioned as a mobile space for work or entertainment, but unfortunately passengers’ engagement in non-driving tasks may exacerbate MS. Finding effective MS countermeasures is crucial to ensure a pleasant passenger experience. Nevertheless, due to the complex mechanism of MS, there are numerous challenges in mitigating it, hindering the development of practical countermeasures. To address this, we first review two prevalent theories explaining the mechanism of MS. Subsequently, this paper provides a summary of current subjective and objective approaches for quantifying motion sickness levels. Then, it surveys existing methods for alleviating MS, including passenger adjustment, intelligent vehicle solutions, and motion cues of various modalities. Furthermore, we outline the limitations and remaining challenges of current research and highlight novel opportunities in the context of intelligent vehicles. Finally, we propose an integrated framework for alleviating MS. The findings of this review will enhance our understanding of carsickness and offer valuable insights for future research and practice in MS mitigation within modern vehicles.

Enhancing perception of vehicle motion by objective positioning of the longitudinal axis of rotation in driving simulators

The automotive industry is heading towards a more objective approach to vehicle testing, but subjective evaluation is still an important part of the development process. Subjective evaluation in physical testing has environmental implications and is dependent on ambient conditions. A more repeatable, faster, safer and more cost-effective tool for subjective evaluation is to use moving base driving simulators. The motion cueing algorithms (MCA) maps the movement of the vehicle into the limited space of the simulator. The choice of reference point, that is, where on the vehicle to sample the motion to feed to the MCA and the alignment of the axis of rotation of the simulator cabin is still an open topic. This paper investigates the choice of reference point and corresponding simulator longitudinal axis of rotation in roll using two methods. The first method uses a linearised model of the combined system of vehicle, simulator and vestibular models. The second method, to position the cabin longitudinal axis of rotation, is based on offline optimisation. The linear model can capture important characteristics of the specific forces and rotations that are fed to the driver through the motion cueing algorithms and offers a method to objectively analyse and potentially tune the motion cueing. The analysis is further complemented with a subjective evaluation of corresponding settings. The results from the linear model, the offline optimisation and the subjective evaluation shows that a reference point at the driver’s head has a clear advantage over the full frequency range compared to a reference point in the chassis roll axis and that the positioning of the cabin longitudinal axis of rotation has a significant effect on the perceived vehicle characteristics.

Smart3DMOT: Smart cascade 3D MOT tracking strategy with motion and appearance association

3D multi-object tracking (MOT) is pivotal for associating the trajectories of objects in autonomous driving. The integration of image and point cloud data in 3D MOT algorithms has emerged as a research hotspot, aiming to achieve both high precision and efficiency tracking. In this paper, we present Smart3DMOT, an novel method that leverages multi-modal detection data and incorporates a cascade matching strategy, intelligently utilizing motion and multi-modal appearance affinity for data association across frames. Our algorithm introduces two association models: the strong motion association model (SM-AM) and the multi-modal fusion feature association model (MMFF-AM), tailored to different tracking scenarios. The SM-AM robustly associates objects by enhancing motion similarity, while the MMFF-AM integrates multi-modal fusion features from images and point clouds to generate appearance similarity between objects and combine motion cues for joint reasoning. Finally, we propose a cascade tracking strategy with a “spatial tracking state confirmer” (STSC). The SM-AM and MMFF-AM models are performed staged matching facilitated by STSC. This intelligent implementation allows SM-AM or MMFF-AM to tackle instances with different tracking difficulties, maintaining the operational efficiency and tracking accuracy of trade-offs. Our approach demonstrates the effectiveness of the proposed modules through comprehensive ablation experiments and the results obtained in the KITTI and nuScenes benchmark show that our method attains state-of-the-art performance in 3D MOT.

Leveraging Tendon Vibration to Enhance Pseudo-Haptic Perceptions in VR.

Pseudo-haptic techniques are used to modify haptic perception by appropriately changing visual feedback to body movements. Based on the knowledge that tendon vibration can affect our somatosensory perception, this article proposes a method for leveraging tendon vibration to enhance pseudo-haptics during free arm motion. Three experiments were performed to examine the impact of tendon vibration on the range and resolution of pseudo-haptics. The first experiment investigated the effect of tendon vibration on the detection threshold of the discrepancy between visual and physical motion. The results indicated that vibrations applied to the inner tendons of the wrist and elbow increased the threshold, suggesting that tendon vibration can augment the applicable visual motion gain by approximately 13% without users detecting the visual/physical discrepancy. Furthermore, the results demonstrate that tendon vibration acts as noise on haptic motion cues. The second experiment assessed the impact of tendon vibration on the resolution of pseudo-haptics by determining the just noticeable difference in pseudo-weight perception. The results suggested that the tendon vibration does not largely compromise the resolution of pseudo-haptics. The third experiment evaluated the equivalence between the weight perception triggered by tendon vibration and that by visual motion gain, that is, the point of subjective equality. The results revealed that vibration amplifies the weight perception and its effect was equivalent to that obtained using a gain of 0.64 without vibration, implying that the tendon vibration also functions as an additional haptic cue. Our results provide design guidelines and future work for enhancing pseudo-haptics with tendon vibration.

High-fidelity learning-based motion cueing algorithm by bypassing worst-case scenario-based tuning technique

The motion cueing algorithm (MCA) enhances the realism of simulator driving experiences by generating vehicle motions within platform limitations. Existing MCAs are typically tuned for worst-case scenarios, limiting their efficiency for medium or slow driving motions. This study proposes a comprehensive MCA unit using learning-based models to overcome this problem and efficiently utilise the simulator workspace for all driving scenarios. Data samples are regenerated to cover various motion signal levels, and three classical washout filters are tuned to extract optimal motion signals. A multilayer perceptron (MLP) is trained with these extracted datasets, forming an AI-based MCA that provides high-fidelity driving motions for any scenario while optimising the platform workspace. Simulink/MATLAB is used for modelling and evaluation. Results demonstrate the proposed model's superior performance, with lower motion sensation errors, a higher correlation between sensed motion signals, and more efficient platform workspace usage.