Abrupt Motion Research Articles

Enhanced neural video compression for cloud gaming videos with aligned frame generation

The burgeoning popularity of cloud gaming makes it critical for efficient video compression to relieve the growing bandwidth pressure. While existing neural video coding approaches have demonstrated strong compression potential on natural videos, there is an absence of efficient neural codecs dedicated to gaming videos. To bridge this gap, in this paper, we propose an end-to-end neural video compression method designed specifically for cloud gaming videos. By effectively utilizing the unique camera motion information inherent to cloud gaming, the previous reconstructed frame is maximally aligned to the current frame through a learning-based module with multiple losses, which then replaces the previous reconstructed frame for optical flow estimation. By significantly reducing the displacement between two consecutive frames caused by camera motion, the motion estimation accuracy is enhanced, effectively handling the large and abrupt motion scenarios frequently present in gaming videos. Furthermore, the aligned tensor obtained in the previous step is used to enhance the latent prior of the entropy model, providing a superior temporal prior for coding. Extensive experimental results demonstrate the superior performance of our proposed method compared to one of the previous state-of-the-art approaches, DCVC-HEM, providing significant progress in end-to-end neural compression in cloud gaming videos.

Corrective mechanisms of motion extrapolation.

Transmission and processing of sensory information in the visual system takes time. For motion perception, our brain can overcome this intrinsic neural delay through extrapolation mechanisms and accurately predict the current position of a continuously moving object. But how does the system behave when the motion abruptly changes and the prediction becomes wrong? Here we address this question by studying the perceived position of a moving object with various abrupt motion changes by human observers. We developed a task in which a bar is monotonously moving horizontally, and then motion suddenly stops, reverses, or disappears-then-reverses around two vertical stationary reference lines. Our results showed that participants overestimated the position of the stopping bar but did not perceive an overshoot in the motion reversal condition. When a temporal gap was added at the reverse point, the perceptual overshoot of the end point scaled with the gap durations. Our model suggests that the overestimation of the object position when it disappears is not linear as a function of its speeds but gradually fades out. These results can thus be reconciled in a single process where there is an interplay of the cortical motion prediction mechanisms and the late sensory transient visual inputs.

Fusion-GRU: A Deep Learning Model for Future Bounding Box Prediction of Traffic Agents in Risky Driving Videos

To ensure the safe and efficient navigation of autonomous vehicles and advanced driving assistance systems in complex traffic scenarios, predicting the future bounding boxes of surrounding traffic agents is crucial. However, simultaneously predicting the future location and scale of target traffic agents from the egocentric view poses challenges because the vehicles’ egomotion causes considerable field-of-view changes. Moreover, in anomalous or risky situations, tracking loss or abrupt motion changes limit the available observation time, requiring learning of cues within a short time window. Existing methods typically use a simple concatenation operation to combine different cues, overlooking their dynamics over time. To address this, this paper introduces the Fusion-Gated Recurrent Unit (Fusion-GRU) network, a novel encoder-decoder architecture for future bounding box localization. Unlike traditional gated recurrent units (GRUs), Fusion-GRU accounts for mutual and complex interactions among input features. Moreover, an intermediary estimator coupled with a self-attention aggregation layer is also introduced to learn sequential dependencies for long-range prediction. Finally, a GRU decoder is employed to predict the future bounding boxes. The proposed method is evaluated on two publicly available datasets, Risky Object Localization and Honda Egocentric Vehicle Intersection. The experimental results showcase the promising performance of the Fusion-GRU, demonstrating its effectiveness in predicting future bounding boxes of traffic agents.

Experimental Investigation of Frictional Resistance in Sliding Contact between Undulating Surfaces and Third-Body Particles

The third-body particle-involved sliding contact between two rough rubbers with wavy surfaces is experimentally studied. The experiment is designed to isolate the direct contact between the first bodies so that friction resistance is induced completely by the interactions between the third-body particle and the surfaces of the rubbers. In dry contact of a single particle, it is found that the particle exhibits pure rolling during the sliding of the first bodies, and the macroscopic friction resistance for overcoming sliding does not depend on the particle size, but it is significantly influenced by the initial position of the surface waviness relative to the particle’s position. The behavior of the particle under lubricated conditions exhibited significant differences. Due to the low local friction at the interface, the particle rapidly glided down to the valley of the waviness during compression. This abrupt motion of the particle resulted in it coming to rest in a stable position, awaiting a substantial force to push it forward. The friction resistance in the case with lubrication was found to be independent of the initial position of the waviness, and its value consistently remained at the maximum found in dry contact. Therefore, lubrication actually increases the macroscopic friction resistance. An approximate solution for the specific case of dry contact is proposed to understand the friction behavior.

Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint

Siamese trackers based on classification and regression have drawn extensive attention due to their appropriate balance between accuracy and efficiency. However, most of them are prone to failure in the face of abrupt motion or appearance changes. This paper proposes a Siamese-based tracker that incorporates spatial-semantic-aware attention and flexible spatiotemporal constraint. First, we develop a spatial-semantic-aware attention model, which identifies the importance of each feature region and channel to target representation through the single convolution attention network with a loss function and increases the corresponding weights in the spatial and channel dimensions to reinforce the target region and semantic information on the target feature map. Secondly, considering that the traditional method unreasonably weights the target response in abrupt motion, we design a flexible spatiotemporal constraint. This constraint adaptively adjusts the constraint weights on the response map by evaluating the tracking result. Finally, we propose a new template updating the strategy. This strategy adaptively adjusts the contribution weights of the tracking result to the new template using depth correlation assessment criteria, thereby enhancing the reliability of the template. The Siamese network used in this paper is a symmetric neural network with dual input branches sharing weights. The experimental results on five challenging datasets show that our method outperformed other advanced algorithms.

High-Performance Detection-Based Tracker for Multiple Object Tracking in UAVs

As a result of increasing urbanization, traffic monitoring in cities has become a challenging task. The use of Unmanned Aerial Vehicles (UAVs) provides an attractive solution to this problem. Multi-Object Tracking (MOT) for UAVs is a key technology to fulfill this task. Traditional detection-based-tracking (DBT) methods begin by employing an object detector to retrieve targets in each image and then track them based on a matching algorithm. Recently, the popular multi-task learning methods have been dominating this area, since they can detect targets and extract Re-Identification (Re-ID) features in a computationally efficient way. However, the detection task and the tracking task have conflicting requirements on image features, leading to the poor performance of the joint learning model compared to separate detection and tracking methods. The problem is more severe when it comes to UAV images due to the presence of irregular motion of a large number of small targets. In this paper, we propose using a balanced Joint Detection and Re-ID learning (JDR) network to address the MOT problem in UAV vision. To better handle the non-uniform motion of objects in UAV videos, the Set-Membership Filter is applied, which describes object state as a bounded set. An appearance-matching cascade is then proposed based on the target state set. Furthermore, a Motion-Mutation module is designed to address the challenges posed by the abrupt motion of UAV. Extensive experiments on the VisDrone2019-MOT dataset certify that our proposed model, referred to as SMFMOT, outperforms the state-of-the-art models by a wide margin and achieves superior performance in the MOT tasks in UAV videos.

Rapid motion estimation and correction using self-encoded FID navigators in 3D radial MRI.

To develop a self-navigated motion compensation strategy for 3D radial MRI that can compensate for continuous head motion by measuring rigid body motion parameters with high temporal resolution from the central k-space acquisition point (self-encoded FID navigator) in each radial spoke. A forward model was created from low-resolution calibration data to simulate the effect of relative motion between the coil sensitivity profiles and the underlying object on the self-encoded FID navigator signal. Trajectory deviations were included in the model as low spatial-order field variations. Three volunteers were imaged at 3 T using a modified 3D gradient-echo sequence acquired with a Kooshball trajectory while performing abrupt and continuous head motion. Rigid body-motion parameters were estimated from the central k-space signal of each spoke using a least-squares fitting algorithm. The accuracy of self-navigated motion parameters was assessed relative to an established external tracking system. Quantitative image quality metrics were computed for images with and without retrospective correction using external and self-navigated motion measurements. Self-encoded FID navigators achieved mean absolute errors of 0.69 ± 0.82 mm and 0.73 ± 0.87° relative to external tracking for maximum motion amplitudes of 12 mm and 10°. Retrospective correction of the 3D radial data resulted in substantially improved image quality for both abrupt and continuous motion paradigms, comparable to external tracking results. Accurate rigid body motion parameters can be rapidly obtained from self-encoded FID navigator signals in 3D radial MRI to continuously correct for head movements. This approach is suitable for robust neuroanatomical imaging in subjects that exhibit patterns of large and frequent motion.

A synergy of the adaptive whale optimization algorithm and differential evolution for abrupt motion tracking

Aiming at the problem that conventional trackers are difficult to adapt to abrupt motion, a tracking algorithm based on a synergy of the adaptive whale optimization algorithm and differential evolution (AWOA-DE) is proposed in this paper. Firstly, the non-linear dynamic feedback strategy based on the one-fifth principle is employed to replace the linear mechanism, which is the variation of exploring step in the basic whale optimization algorithm (WOA), and an adaptive WOA (AWOA) is generated to improve the global optimization ability of the WOA. Secondly, AWOA adopts a random sampling strategy to generate new samples, which could ensure the diversity of solutions, but may lead to instability. By mixing AWOA and DE algorithms, the population is divided into two groups, which are updated by AWOA and DE algorithms respectively, to ensure the diversity of the populations and improve the optimization accuracy of the algorithm. Finally, a visual tracker based on AWOA-DE is proposed, which can deal with the problem of abrupt motion in tracking. In addition, the effectiveness of the hybrid optimization algorithm is evaluated using 23 benchmark functions. In the experiment of visual tracking, extensive experimental results show that the AWOA-DE tracker (AWOADET) has much stronger competitiveness compared to the WOA tracker (WOAT) and the other 10 state-of-the-art trackers, especially for abrupt motion tracking.

Detection of Torque Security Problems Based on the Torsion of Side Shafts in Electrified Vehicles

In the case of electric vehicle drives, faults in the drive system or in the traction inverter, which controls the vehicle drive unit, could lead to abrupt and unpredictable motion as well as acceleration of the vehicle. In terms of functional safety, the typically existing, permanent mechanical connection of the drive machine with the drive wheels poses a high safety risk. In particular, unintended motion of the vehicle from a standstill is especially critical due to the high risk of injury to traffic participants. To reduce this risk, appropriate monitoring algorithms can be applied for the rapid detection of faulty operation. A corresponding algorithm for fault detection in the electric drive of a vehicle is presented in this paper. In addition to the description of the algorithms, various driving maneuvers of an electric single-wheel drivetrain are simulated in fault-free and faulty operation on a hardware-in-the-loop test bench. The focus here is on the consideration of driving-off operations.

Deep learning based registration for head motion correction in positron emission tomography as a strategy for improved image quantification

Objectives: Positron emission tomography (PET) is affected by various kinds of patient movement during a scan. Frame-by-frame image registration is one of the most practiced motion correction techniques. In recent years, deep learning has shown a remarkable ability to quickly and accurately register images once trained. This paper studies the feasibility of using a deep learning framework to correct 3D positron emission tomography image volumes for head motion in routine positron emission tomography imaging to improve quantification in motion impacted data.Materials and Methods: A neural network was trained with 3D positron emission tomography image volumes in an unsupervised manner to predict transformation parameters required to perform image registration. A multi-step convolutional neural network (CNN) was combined with a spatial transform layer. Pairs of target and source images were used as input to the network. To prepare the training dataset, a previously published TOF-PEPT algorithm was applied to automatically detect static frames where the patient remained in a relatively steady position and transitional frames where they underwent abrupt motion. A single image volume was reconstructed for each static frame. The image reconstructed from the first static frame served as the target image with images from subsequent static frames being used as source images. The trained neural network predicted transformation parameters that could be used to perform frame-by-frame image-based motion correction but also enabled raw listmode positron emission tomography data correction where individual line-of-responses were repositioned. Line profiles and ROIs were drawn across the reconstructed image volumes to compare performance and quantitative results between standard registration tools and the deep learning technique. Corrected volumes were further compared to motion free images quantitatively using Dice indices.Results: In total, one hundred 3D positron emission tomography image volumes were used to train the network. Cross-validation was carried out using a 4:1 ratio for the training and test data. A conventional algorithm for affine registration from the Advanced Normalization Tools (ANTs) software package served as a baseline. To evaluate the correction performance, the mean Dice index and standardized uptake value (SUV) were used. Application of the algorithm to clinical data showed good performance with respect to registration accuracy as well as processing time. The neural network yielded a mean Dice index of ∼0.87 which was similar to the advanced Normalization Tools algorithm and did so ∼3x faster using a multi-core CPU and ∼20x faster with a GPU. Standardized uptake value analysis showed that quantitative results were 30%–60% higher in the motion-corrected images, and the neural network performed better than or close to the advanced Normalization Tools.Conclusion: The aim of this work was to study the quantitative impact of using a data-driven deep learning motion correction technique for positron emission tomography data and assess its performance. The results showed the technique is capable of producing high quality registrations that compensate for patient motion that occurs during a scan and improve quantitative accuracy.

The collective burst mechanism of angular jumps in liquid water

Understanding the microscopic origins of collective reorientational motions in aqueous systems requires techniques that allow us to reach beyond our chemical imagination. Herein, we elucidate a mechanism using a protocol that automatically detects abrupt motions in reorientational dynamics, showing that large angular jumps in liquid water involve highly cooperative orchestrated motions. Our automatized detection of angular fluctuations, unravels a heterogeneity in the type of angular jumps occurring concertedly in the system. We show that large orientational motions require a highly collective dynamical process involving correlated motion of many water molecules in the hydrogen-bond network that form spatially connected clusters going beyond the local angular jump mechanism. This phenomenon is rooted in the collective fluctuations of the network topology which results in the creation of defects in waves on the THz timescale. The mechanism we propose involves a cascade of hydrogen-bond fluctuations underlying angular jumps and provides new insights into the current localized picture of angular jumps, and its wide use in the interpretations of numerous spectroscopies as well in reorientational dynamics of water near biological and inorganic systems. The role of finite size effects, as well as of the chosen water model, on the collective reorientation is also elucidated.

Abrupt moving target tracking based on quantum enhanced particle filter

Abrupt-motion tracking is challenging due to the target’s unpredictable action. Although particle filter (PF) is suitable for target tracking of nonlinear non-Gaussian systems, it suffers from the problems of particle impoverishment and sample-size dependency. This paper proposed a quantum-inspired particle filter for abrupt-motion tracking. We apply the concept of quantum superposition to transform classical particles into quantum particles. Quantum representation and corresponding quantum operations are addressed to utilize quantum particles. The superposition property of quantum particles avoids the concerns of particle impoverishment and sample-size dependency. The proposed diversity-preserving quantum-enhanced particle filter (DQPF) obtains better accuracy and stability with fewer particles. A smaller sample size also helps to reduce computational complexity. Moreover, it has significant advantages for abrupt-motion tracking. The quantum particles are propagated at the prediction stage. They will exist at possible places when abrupt motion occurs, which reduces the tracking delay and enhances the tracking accuracy. This paper conducted experiments compared to state-of-the-art particle filter algorithms. The numerical results demonstrate that the DQPF is not susceptible to motion mode and particle number. Meanwhile, DQPF maintains excellent accuracy and stability.

Enhancing Conventional Geometry-Based Visual Odometry Pipeline Through Integration of Deep Descriptors

Geometry-based Visual Odometry (VO) techniques are renowned in the fields of computer vision and robotics. They use methods from multi-view geometry to estimate camera motion from visual data obtained from one or more cameras. Tracking the camera motion precisely between different views is dependent on the correct estimation of correspondences between salient points of the views. In practice, geometry-based methods are found to be quite effective but do not perform well in challenging cases due to tracking failures caused by abrupt motion, occlusions, textureless and low-light scenes, etc. On the contrary, end-to-end learning from visual data using deep neural networks is an emerging area of research and deals with challenging cases successfully. Despite being computationally expensive, these methods do not outperform their counterparts in conditions favorable to geometry-based methods. Considering these facts in this work, our goal is to integrate deep descriptors to improve the correspondence between image points for tracking in a traditional geometry-based VO pipeline. We propose a simple stereo VO pipeline inspired by popular techniques found in the literature. Two conventional and four deep descriptors have been used in our experiments conducted on various image sequences of the challenging KITTI benchmark dataset. We have determined empirically that deep descriptors can effectively minimize drift in the VO estimates and produce better camera trajectories. The experimental results on the KITTI dataset demonstrate that our VO method performs at par with the state-of-the-art works reported in the literature.

Tele‐aiming control design for reconnaissance robot using a strong tracking multi‐model extended super‐twisting observer

AbstractIn this study, a tele‐aiming control system for reconnaissance robots is proposed, capable of enabling operators to quickly control the robot to aim at and track the manoeuvring target with random motion characteristics under measurement noise. To track such manoeuvring targets, a cascaded multi‐model super‐twisting observer is designed. First, the system uncertainty and the inertial impact force disturbance generated during rapid tele‐aiming are expanded into a lumped disturbance state. Subsequently, a strong tracking filter is employed to filter and estimate the system output and the lumped disturbance, and the output of the filter is input into the multi‐model extended state super‐twisting observer. The final second‐order state estimation and lumped disturbance estimation of the augmented system are conducted in accordance with the weighted sum of all super‐twisting observers. Lastly, a non‐singular terminal sliding mode controller is designed to compensate for the lumped disturbance and achieve the fast trajectory tracking. To verify the effectiveness of the proposed observer‐based controller, a series of manoeuvring target tracking experiments was performed. As revealed by the experimental results, the proposed controller can quickly control the reconnaissance robot to aim at and track the manoeuvring target with random abrupt motion characteristics under measurement noise disturbance.

An objective evaluation method for head motion estimation in PET—Motion corrected centroid-of-distribution

Head motion presents a continuing problem in brain PET studies. A wealth of motion correction (MC) algorithms had been proposed in the past, including both hardware-based methods and data-driven methods. However, in most real brain PET studies, in the absence of ground truth or gold standard of motion information, it is challenging to objectively evaluate MC quality. For MC evaluation, image-domain metrics, e.g., standardized uptake value (SUV) change before and after MC are commonly used, but this measure lacks objectivity because 1) other factors, e.g., attenuation correction, scatter correction and parameters used in the reconstruction, will confound MC effectiveness; 2) SUV only reflects final image quality, and it cannot precisely inform when an MC method performed well or poorly during the scan time period; 3) SUV is tracer-dependent and head motion may cause increases or decreases in SUV for different tracers, so evaluating MC effectiveness is complicated. Here, we present a new algorithm, i.e., motion corrected centroid-of-distribution (MCCOD) to perform objective quality control for measured or estimated rigid motion information. MCCOD is a three-dimensional surrogate trace of the center of tracer distribution after performing rigid MC using the existing motion information. MCCOD is used to inform whether the motion information is accurate, using the PET raw data only, i.e., without PET image reconstruction, where inaccurate motion information typically leads to abrupt changes in the MCCOD trace. MCCOD was validated using simulation studies and was tested on real studies acquired from both time-of-flight (TOF) and non-TOF scanners. A deep learning-based brain mask segmentation was implemented, which is shown to be necessary for non-TOF MCCOD generation. MCCOD is shown to be effective in detecting abrupt translation motion errors in slowly varying tracer distribution caused by the motion tracking hardware and can be used to compare different motion estimation methods as well as to improve existing motion information.

Hybrid modified marine predators algorithm with teaching-learning-based optimization for global optimization and abrupt motion tracking

Hybrid modified marine predators algorithm with teaching-learning-based optimization for global optimization and abrupt motion tracking

Clinical implications of compromised 82Rb PET data acquisition

BackgroundWe wished to document the prevalence and quantitative effects of compromised 82Rb PET data acquisitions on myocardial flow reserve (MFR). Methods and ResultsData were analyzed retrospectively for 246 rest and regadenoson-stress studies of 123 patients evaluated for known or suspected CAD. An automated injector delivered pre-determined activities of 82Rb. Automated quality assurance algorithms identified technical problems for 7% (9/123) of patients. Stress data exhibited 2 instances of scanner saturation, 1 blood peak detection, 1 blood peak width, 1 gradual patient motion, and 2 abrupt patient motion problems. Rest data showed 1 instance of blood peak width and 2 abrupt patient motion problems. MFR was lower for patients with technical problems flagged by the quality assurance algorithms than those without technical problems (1.5 ± 0.5 versus 2.1 ± 0.7, P = 0.01), even though rest and stress ejection fraction, asynchrony and relative myocardial perfusion measures were similar for these two groups (P > 0.05), suggesting that MFR accuracy was adversely affected by technical errors. ConclusionIt is important to verify integrity of 82Rb data to ensure MFR computation quality.

Motion detection and correction for carotid MRI using a markerless optical system

PurposeMotion related artifact is a challenge for MRI, especially when imaging regions like the carotid artery where complex motion (abrupt and bulk motion) may occur. This study aims to develop a non-contact motion detection and correction system for carotid MRI using a markerless optical tracking system. MethodsThe proposed markerless optical tracking system consisted of a cross-line laser, an MRI-compatible camera and plastic holders mounted inside the scanner bore. The neck motion of the subject can be captured by monitoring the change of the projected laser position in real-time. The system was used to correct both abrupt motion and bulk motion for carotid MRI. The abrupt motion (e.g. coughing) was compensated by discarding the corrupted k-space lines and re-estimating the missing lines using SPIRiT algorithm. The bulk motion was corrected by phase adjustment of k-space lines according to the measured 1D-translational bulk motion (along anterior-posterior direction) and optimized in-plane translation parameters. Ten volunteers underwent carotid MRI with real-time neck motion detection and retrospective motion correction. Artery sharpness, vessel wall thickness and overall image quality score were compared between the motion-corrupted image and motion-corrected images of different correction strategies. ResultsBoth the abrupt motion and the bulk motion during carotid scanning were successfully detected and corrected. The results of ten volunteers demonstrated significant improvement in carotid artery sharpness, vessel wall thickness measurement, and overall image quality score using the proposed markerless optical tracking system and motion correction strategies. ConclusionThe proposed markerless structured light based motion detection and correction system can sensitively detect both abrupt and bulk motion during carotid MR scans. By correcting for both abrupt and bulk motion, vessel wall delineation was improved in carotid MR images, which could potentially facilitate carotid plaque identification and atherosclerosis diagnosis in the future.

CORB2I-SLAM: An Adaptive Collaborative Visual-Inertial SLAM for Multiple Robots

The generation of robust global maps of an unknown cluttered environment through a collaborative robotic framework is challenging. We present a collaborative SLAM framework, CORB2I-SLAM, in which each participating robot carries a camera (monocular/stereo/RGB-D) and an inertial sensor to run odometry. A centralized server stores all the maps and executes processor-intensive tasks, e.g., loop closing, map merging, and global optimization. The proposed framework uses well-established Visual-Inertial Odometry (VIO), and can be adapted to use Visual Odometry (VO) when the measurements from inertial sensors are noisy. The proposed system solves certain disadvantages of odometry-based systems such as erroneous pose estimation due to incorrect feature selection or losing track due to abrupt camera motion and provides a more accurate result. We perform feasibility tests on real robot autonomy and extensively validate the accuracy of CORB2I-SLAM on benchmark data sequences. We also evaluate its scalability and applicability in terms of the number of participating robots and network requirements, respectively.

How similar are responses to background motion and target displacements?

When making a goal-directed movement towards a target, our hand follows abrupt background motion. This response resembles that of a shift in the target’s position. Does background motion simply change the position towards which the movement is guided? If so, the response to background motion should resemble the response to a target displacement. To find out whether this is the case, we ran two exploratory studies where we asked participants to hit a moving target at a specified moment. At various times during the hand’s movement, the background could move briefly at one of several speeds, and for various durations. The response to abrupt background motion was larger when the background moved later in the movement and when the background moved faster, in line with known responses to target displacements. The response to a second epoch of background motion was smaller than it would have been if there had been no first epoch, in contrast to responses to multiple target displacements. If the background was already moving before the target appeared, the hand even moved in the opposite direction. Thus, the response to background motion and that to a target displacement are clearly not identical, but they do share several features.