Learning-based Motion Research Articles

Deep Learning Detection of Hand Motion During Microvascular Anastomosis Simulations Performed by Expert Cerebrovascular Neurosurgeons

Deep learning enables precise hand tracking without the need for physical sensors, allowing for unsupervised quantitative evaluation of surgical motion and tasks. We quantitatively assessed the hand motions of experienced cerebrovascular neurosurgeons during simulated microvascular anastomosis using deep learning. We explored the extent to which surgical motion data differed among experts. A deep learning detection system tracked 21 landmarks corresponding to digit joints and the wrist on each hand of 5 expert cerebrovascular neurosurgeons. Tracking data for each surgeon were analyzed over long and short time intervals to examine gross movements and micromovements, respectively. Quantitative algorithms assessed the economy and flow of motion by calculating mean movement distances from the baseline median landmark coordinates and median times between sutures, respectively. Tracking data correlated with specific surgical actions observed in microanastomosis video analysis. Economy of motion during suturing was calculated as 19, 26, 29, 27, and 28 pixels for surgeons 1, 2, 3, 4, and 5, respectively. Flow of motion during microanastomosis was 31.96, 29.40, 28.90, 7.37, and 47.21seconds for surgeons 1, 2, 3, 4, and 5, respectively. Hand tracking data showed similarities among experts, with low movements from baseline, minimal excess motion, and rhythmic suturing patterns. The data revealed unique patterns related to each expert's habits and techniques. The results showed that surgical motion can be correlated with hand motion and assessed using mathematical algorithms. We also demonstrated the feasibility and potential of deep learning-based motion detection to enhance surgical training.

High-fidelity learning-based motion cueing algorithm by bypassing worst-case scenario-based tuning technique

The motion cueing algorithm (MCA) enhances the realism of simulator driving experiences by generating vehicle motions within platform limitations. Existing MCAs are typically tuned for worst-case scenarios, limiting their efficiency for medium or slow driving motions. This study proposes a comprehensive MCA unit using learning-based models to overcome this problem and efficiently utilise the simulator workspace for all driving scenarios. Data samples are regenerated to cover various motion signal levels, and three classical washout filters are tuned to extract optimal motion signals. A multilayer perceptron (MLP) is trained with these extracted datasets, forming an AI-based MCA that provides high-fidelity driving motions for any scenario while optimising the platform workspace. Simulink/MATLAB is used for modelling and evaluation. Results demonstrate the proposed model's superior performance, with lower motion sensation errors, a higher correlation between sensed motion signals, and more efficient platform workspace usage.

A deep blur learning-based motion artifact reduction algorithm for dental cone-beam computed tomography images

We propose a motion artifact correction algorithm (DMBL) for reducing motion artifacts in reconstructed dental cone-beam computed tomography (CBCT) images based on deep blur learning. A blur encoder was used to extract motion-related degradation features to model the degradation process caused by motion, and the obtained motion degradation features were imported in the artifact correction module for artifact removal. The artifact correction module adopts a joint learning framework for image blur removal and image blur simulation for treatment of spatially varying and random motion patterns. Comparative experiments were conducted to verify the effectiveness of the proposed method using both simulated motion data sets and clinical data sets. The experimental results with the simulated dataset showed that compared with the existing methods, the PSNR of the proposed method increased by 2.88%, the SSIM increased by 0.89%, and the RMSE decreased by 10.58%. The results with the clinical dataset showed that the proposed method achieved the highest expert level with a subjective image quality score of 4.417 (in a 5-point scale), significantly higher than those of the comparison methods. The proposed DMBL algorithm with a deep blur joint learning network structure can effectively reduce motion artifacts in dental CBCT images and achieve high-quality image restoration.

Safe Reinforcement Learning-Based Motion Planning for Functional Mobile Robots Suffering Uncontrollable Mobile Robots

Safe Reinforcement Learning-Based Motion Planning for Functional Mobile Robots Suffering Uncontrollable Mobile Robots

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

An Ultralow-Power Real-Time Machine Learning Based fNIRS Motion Artifacts Detection.

Due to iterative matrix multiplications or gradient computations, machine learning modules often require a large amount of processing power and memory. As a result, they are often not feasible for use in wearable devices, which have limited processing power and memory. In this study, we propose an ultralow-power and real-time machine learning-based motion artifact detection module for functional near-infrared spectroscopy (fNIRS) systems. We achieved a high classification accuracy of 97.42%, low field-programmable gate array (FPGA) resource utilization of 38354 lookup tables and 6024 flip-flops, as well as low power consumption of 0.021 W in dynamic power. These results outperform conventional CPU support vector machine (SVM) methods and other state-of-the-art SVM implementations. This study has demonstrated that an FPGA-based fNIRS motion artifact classifier can be exploited while meeting low power and resource constraints, which are crucial in embedded hardware systems while keeping high classification accuracy.

A machine learning-based motion training approach applied to multilegged and bipedal robots

To promptly gain an understanding of disasters as they occur and to draft plans for search and rescue operations, various types of robots are used. Robots not only increase rescue efficiency but also reduce firefighter casualties. Therefore, this study investigated how bionic robots can be utilized to search destroyed and chaotic disaster sites as quickly as possible. However, manually adjusting the motion parameters of robots performing robot motions is extremely inefficient. To resolve this problem, this study used machine learning algorithms to allow robots to train themselves in the environment and autonomously determine their optimal motion parameters. However, many types of machine learning algorithms exist, each with their own strengths and weaknesses. Therefore, this study designed a series of experiments to investigate the features of each algorithm in optimizing the robots’ motions; subsequently, this study compared the strengths and weaknesses of each algorithm based on their performance. The results indicated that for both multipedal and bipedal robots, the use of machine learning to find the optimal motion parameters is both feasible and practical.

Reinforcement learning-based motion planning in partially observable environments under ethical constraints

Reinforcement learning-based motion planning in partially observable environments under ethical constraints

Reinforcement learning-based motion control for snake robots in complex environments

AbstractSnake robots can move flexibly due to their special bodies and gaits. However, it is difficult to plan their motion in multi-obstacle environments due to their complex models. To solve this problem, this work investigates a reinforcement learning-based motion planning method. To plan feasible paths, together with a modified deep Q-learning algorithm, a Floyd-moving average algorithm is proposed to ensure smoothness and adaptability of paths for snake robots’ passing. An improved path integral algorithm is used to work out gait parameters to control snake robots to move along the planned paths. To speed up the training of parameters, a strategy combining serial training, parallel training, and experience replaying modules is designed. Moreover, we have designed a motion planning framework consists of path planning, path smoothing, and motion planning. Various simulations are conducted to validate the effectiveness of the proposed algorithms.

An Unsupervised Deep Learning Framework for Retrospective Gating of Catheter-Based Cardiac Imaging

Motion artifacts are a major challenge in the in vivo application of catheter-based cardiac imaging modalities. Gating is a critical tool for suppressing motion artifacts. Electrocardiogram (ECG) gating requires a trigger device or synchronous ECG recordings for retrospective analysis. Existing retrospective software gating methods extract gating signals through separate steps based on changes in vessel morphology or image features, which require a high computational cost and are prone to error accumulation. In this paper, we report on an end-to-end unsupervised learning framework for retrospective image-based gating (IBG) of catheter-based intracoronary images, named IBG Network. It establishes a direct mapping from a continuously acquired image sequence to a gated subsequence. The network was trained on clinical data sets in an unsupervised manner, addressing the difficulty of obtaining the gold standard in deep learning-based motion suppression techniques. Experimental results of in vivo intravascular ultrasound and optical coherence tomography sequences show that the proposed method has better performance in terms of motion artifact suppression and processing efficiency compared with the state-of-the-art nonlearning signal-based and IBG methods.

Deep Reinforcement Learning-Based Motion Control for Unmanned Vehicles from the Perspective of Multi-Sensor Data Fusion

In this paper, the vehicle position points obtained by multi-sensor fusion are taken as the observed values, and Kalman filter is combined with the vehicle kinematics equation to further improve the vehicle trajectory. To realize this, mathematical principles of deep reinforcement learning are analyzed, and the theoretical basis of reinforcement learning is also analyzed. It is proved that the controller based on dynamic model is better than the controller based on kinematics in deviation control, and the performance of the controller based on deep reinforcement learning is also verified. The simulation data show that the proportion integration differentiation (PID) controller has a better tracking effect, but it does not have the constraint ability, which leads to radical acceleration change, resulting in unstable acceleration and deceleration control. Therefore, the deep reinforcement learning controller is selected as the longitudinal velocity tracking controller. The effectiveness of lateral and longitudinal motion decoupling strategy is verified by simulation experiments.

Unsupervised motion artifact correction of turbo spin-echo MRI using deep image prior.

In MRI, motion artifacts can significantly degrade image quality. Motion artifact correction methods using deep neural networks usually required extensive training on large datasets, making them time-consuming and resource-intensive. In this paper, an unsupervised deep learning-based motion artifact correction method for turbo-spin echo MRI is proposed using the deep image prior framework. The proposed approach takes advantage of the high impedance to motion artifacts offered by the neural network parameterization to remove motion artifacts in MR images. The framework consists of parameterization of MR image, automatic spatial transformation, and motion simulation model. The proposed method synthesizes motion-corrupted images from the motion-corrected images generated by the convolutional neural network, where an optimization process minimizes the objective function between the synthesized images and the acquired images. In the simulation study of 280 slices from 14 subjects, the proposed method showed a significant increase in the averaged structural similarity index measure by 0.2737 in individual coil images and by 0.4550 in the root-sum-of-square images. In addition, the ablation study demonstrated the effectiveness of each proposed component in correcting motion artifacts compared to the corrected images produced by the baseline method. The experiments on real motion dataset has shown its clinical potential. The proposed method exhibited significant quantitative and qualitative improvements in correcting rigid and in-plane motion artifacts in MR images acquired using turbo spin-echo sequence.

Deep MARL-Based Resilient Motion Planning for Decentralized Space Manipulator

Space manipulators play an important role in the on-orbit services and planetary surface operation. In the extreme environment of space, space manipulators are susceptible to a variety of unknown disturbances. How to have a resilient guarantee in failure or disturbance is the core capability of its future development. Compared with traditional motion planning, learning-based motion planning has gradually become a hot spot in current research. However, no matter what kind of research ideas, the single robotic manipulator is studied as an independent agent, making it unable to provide sufficient flexibility under conditions such as external force disturbance, observation noise, and mechanical failure. Therefore, this paper puts forward the idea of “discretization of the traditional single manipulator”. Different discretization forms are given through the analysis of the multi-degree-of-freedom single-manipulator joint relationship, and a single-manipulator representation composed of multiple new subagents is obtained. Simultaneously, to verify the ability of the new multiagent representation to deal with interference, we adopted a centralized multiagent reinforcement learning framework. The influence of the number of agents and communication distances on learning-based planning results is analyzed in detail. In addition, by imposing joint locking failures on the manipulator and introducing observation and action interference, it is verified that the “multiagent robotic manipulator” obtained after discretization has stronger antidisturbance resilient capability than the traditional single manipulator.

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

Deep Learning-Based Motion Prediction Leveraging Autonomous Driving Datasets: State-of-the-Art

Deep Learning-Based Motion Prediction Leveraging Autonomous Driving Datasets: State-of-the-Art

Machine learning-based motion tracking reveals an inverse correlation between adhesivity and surface motility of the leptospirosis spirochete

Bacterial motility is often a crucial virulence factor for pathogenic species. A common approach to study bacterial motility is fluorescent labeling, which allows detection of individual bacterial cells in a population or in host tissues. However, the use of fluorescent labeling can be hampered by protein expression stability and/or interference with bacterial physiology. Here, we apply machine learning to microscopic image analysis for label-free motion tracking of the zoonotic bacterium Leptospira interrogans on cultured animal cells. We use various leptospiral strains isolated from a human patient or animals, as well as mutant strains. Strains associated with severe disease, and mutant strains lacking outer membrane proteins (OMPs), tend to display fast mobility and reduced adherence on cultured kidney cells. Our method does not require fluorescent labeling or genetic manipulation, and thus could be applied to study motility of many other bacterial species.

Deep learning-based motion compensation for automotive SAR imaging

In this paper, we propose an unsupervised image-to-image translation (UNIT) network as a deep learning-based motion compensation method to compensate for phase errors that may occur when acquiring data from a synthetic aperture radar (SAR) system. In general, in the SAR system measurement process, a problem arises in obtaining data containing phase errors due to the platform’s non-ideal path or unstable pose. Therefore, we propose a deep learning-based motion compensation method and verify the performance of the proposed method through an experiment using an automotive radar. As a result of comparing the performance with existing motion compensation methods based on actual measurement data, the proposed UNIT network showed 8.68% higher peak signal-to-noise ratio (PSNR) and 11.66% higher structural similarity index measure (SSIM) performance than phase gradient autofocus, and 7.98% higher PSNR and 10.82% higher SSIM performance than minimum entropy autofocus.

Learning based motion artifacts processing in fNIRS: a mini review.

This paper provides a concise review of learning-based motion artifacts (MA) processing methods in functional near-infrared spectroscopy (fNIRS), highlighting the challenges of maintaining optimal contact during subject movement, which can lead to MA and compromise data integrity. Traditional strategies often result in reduced reliability of the hemodynamic response and statistical power. Recognizing the limited number of studies focusing on learning-based MA removal, we examine 315 studies, identifying seven pertinent to our focus area. We discuss the current landscape of learning-based MA correction methods and highlight research gaps. Noting the absence of standard evaluation metrics for quality assessment of MA correction, we suggest a novel framework, integrating signal and model quality considerations and employing metrics like ΔSignal-to-Noise Ratio (ΔSNR), confusion matrix, and Mean Squared Error. This work aims to facilitate the application of learning-based methodologies to fNIRS and improve the accuracy and reliability of neurovascular studies.

Learning for Depth Control of a Robotic Penguin: A Data-Driven Model Predictive Control Approach

For bionic underwater robots, it is a great challenge for depth control due to model uncertainty and strong nonlinearity. To this end, we propose a data-driven model predictive control (MPC) approach using reinforcement learning (RL) for robotic penguin depth control. First, by imitating the underwater mode of the biological penguin, a robotic prototype with a tendon-driven head, two degrees of freedom wings, and a tendon-driven tail was designed. Then, a data-driven MPC framework is proposed considering the structure and motion properties of the robotic penguin. Specially, a data-based learning environment is constructed using a motion capture system, computational fluid dynamics, and a back-propagation neural network. Meanwhile, to maximize the benefits of the controller while ensuring safety and stability, a data-driven MPC using the RL scheme is applied to approximate the optimal policy. Combined with an appropriate reward design and periodic training, the closed-loop controller performance is significantly improved, and the validity of the proposed framework is finally tested by extensive simulations and experiments. Notably, this work will provide valuable insights into the learning-based motion control of bionic underwater robots.

Unrolled and rapid motion-compensated reconstruction for cardiac CINE MRI

In recent years Motion-Compensated MR reconstruction (MCMR) has emerged as a promising approach for cardiac MR (CMR) imaging reconstruction. MCMR estimates cardiac motion and incorporates this information in the reconstruction. However, two obstacles prevent the practical use of MCMR in clinical situations: First, inaccurate motion estimation often leads to inferior CMR reconstruction results. Second, the motion estimation frequently leads to a long processing time for the reconstruction. In this work, we propose a learning-based and unrolled MCMR framework that can perform precise and rapid CMR reconstruction. We achieve accurate reconstruction by developing a joint optimization between the motion estimation and reconstruction, in which a deep learning-based motion estimation framework is unrolled within an iterative optimization procedure. With progressive iterations, a mutually beneficial interaction can be established in which the reconstruction quality is improved with more accurate motion estimation. Further, we propose a groupwise motion estimation framework to speed up the MCMR process. A registration template based on the cardiac sequence average is introduced, while the motion estimation is conducted between the cardiac frames and the template. By applying this framework, cardiac sequence registration can be accomplished with linear time complexity. Experiments on 43 in-house acquired 2D CINE datasets indicate that the proposed unrolled MCMR framework can deliver artifacts-free motion estimation and high-quality CMR reconstruction even for imaging acceleration rates up to 20x. We compare our approach with state-of-the-art reconstruction methods and it outperforms them quantitatively and qualitatively in all adapted metrics across all acceleration rates.