3D Motion Research Articles

Detecting and Tracking 6-DoF Motion of Unknown Dynamic Objects in Industrial Environments Using Stereo Visual Sensing

Detecting and Tracking 6-DoF Motion of Unknown Dynamic Objects in Industrial Environments Using Stereo Visual Sensing

The Use of Digital Media Arts in Animation Filmmaking in the Age of Digital Intelligence

Abstract The development of digital media art has brought a broader application prospect for the production of digital animation movies. The incorporation of Digital Media Arts into Animation Filmmaking during the Age of Digital Intelligence The DTW algorithm is selected to temporally regularize and align the motion data of the animated character, and the motion model is downsized and optimized by combining with the PCA method, to obtain the motion sequence with the characteristics of the animated character itself. The multi-scale method is utilized to decompose the original motion data when combined with motion capture technology to establish a 3D motion database. Analyze the motion connection curves between different video clips and the generated animation data after and before motion modeling using the 3D motion database of broadcast gymnastics. Analyzing the drawing efficiency of virtual character models is done using an ordinary human character model with different voxel accuracy. Before the motion library modeling, the average joint error measured in degrees was 10.012°, as indicated by the data. The modeling procedure results in a reduction of the average joint error to 4.281°. The accuracy of animated character generation motion can be improved by using motion modeling. Character motion generation suggestions for animated movie production are provided in this study.

Kinematic and Kinetic Gait Features Associated with Mild Cognitive Impairment in Parkinson's Disease.

Mild cognitive impairment (MCI) and gait deficits are commonly associated with Parkinson's disease (PD). Early detection of MCI associated with Parkinson's disease (PD-MCI) and its biomarkers is critical to managing disability in PD patients, reducing caregiver burden and healthcare costs. Gait is considered a surrogate marker for cognitive decline in PD. However, gait kinematic and kinetic features in PD-MCI patients remain unknown. This study was designed to explore the difference in gait kinematics and kinetics during single-task and dual-task walking between PD patients with and without MCI. Kinematic and kinetic data of 90 PD patients were collected using 3D motion capture system. Differences in gait kinematic and kinetic gait features between groups were identified by using: first, univariate statistical analysis and then a supervised machine learning analysis. The findings of this study showed that the presence of MCI in PD patients is coupled with kinematic and kinetic deviations of gait cycle which may eventually identify two different phenotypes of the disease. Indeed, as shown by the demographical and clinical comparison between the two groups, PD-MCI patients were older and more impaired. Moreover, PD-MCI kinematic results showed that cognitive dysfunction coexists with more severe axial symptoms and an increase postural flexion. A lack of physiological distal-to-proximal shift in joint kinetics was evidenced in the PD phenotype associated with cognitive impairments.

Research on 3D Animation Capture Driving Technology for Digital Media

Abstract In recent years, the extensive use of personalized cartoon models in film and television entertainment, games, and other fields has made 3D animation capture drive technology an important research topic in the field of virtual reality. This paper focuses on research in the field of 3D animation capture technology, specifically from two perspectives: human movement and facial expression. The human body movement node data are collected using sensors, and the movement state is represented in three-dimensional space using the form of quaternion, and the Euler angle and rotation matrix are applied to realize the data conversion, respectively. Personalized models of human facial expression data are acquired and built using optical motion capture technology. Enter the data into the database, finish the dynamic 3D re-modeling after data segmentation, and implement 3D animation utilizing 3D motion capture driving technology in 3D animation. The facial fluency index based on the sensor in human body movement with optical capture is about 80, and the fluency index is higher than 100 in four frames of 105, 110, 127, and 128. The mean value of the animated movie designed based on 3D animation capture driving technology is higher than that of the control group in the four aspects of interactivity, interestingness, informativeness, and behavioral change, and the differences between the two groups are 3.1977, 1.899, 0.4378, 1.1444, and the mean value is less than 0.01, the animated movie designed based on 3D motion capture technology brings better viewing experience to the audience.

Numerical Solution for Solving Procedure for 3D Motions Near Libration Points in the Circular Restricted Three Body Problem (CR3BP)

Numerical Solution for Solving Procedure for 3D Motions Near Libration Points in the Circular Restricted Three Body Problem (CR3BP)

Multiplanar lumbar, pelvis and kick leg sequencing during soccer instep kicking from different approach angles

Multiplanar kinematic and kinetic sequencing from different approach angles can highlight how soccer players perform fast and accurate kicks. This study therefore aimed to a) determine multiplanar torso, pelvis and kick leg sequencing during instep kicks and b) highlight the effect of different approach angles on these sequencing patterns. Twenty male soccer players (mass 77.9 ± 6.5 kg, height 1.71 ± 0.09 m, age 23.2 ± 3.7 years) performed kicks from self-selected (∼30−45°), straight (0°) and wide (67.5°) approaches and multiplanar lumbo-pelvic, hip and knee angular velocities, moments and powers were derived from 3D motion analysis. The results suggest tension arc release between the upper and lower body functions as a two-stage mechanism. The first phase of arc release was characterised by increases in concentric hip flexion and transverse lumbo-pelvic velocities towards the ball. The second phase was characterised by increasing concentric lumbo-pelvic flexion and knee extension work to angularly accelerate the kicking knee towards foot-to-ball contact. Further, alterations in kinematic and kinetic sequencing helped maintain performance (ball and foot velocities at ball contact) and accuracy at approach angles other than self-selected. These findings can help coaches and practitioners design effective training practices.

Three-Dimensional Motion Tracking of High-Speed Magnetic Targets Using Microwave-Free Quantum Imaging Technology

Three-Dimensional Motion Tracking of High-Speed Magnetic Targets Using Microwave-Free Quantum Imaging Technology

Identity-Aware and Shape-Aware Propagation of Face Editing in Videos.

The development of deep generative models has inspired various facial image editing methods, but many of them are difficult to be directly applied to video editing due to various challenges ranging from imposing 3D constraints, preserving identity consistency, ensuring temporal coherence, etc. To address these challenges, we propose a new framework operating on the StyleGAN2 latent space for identity-aware and shape-aware edit propagation on face videos. In order to reduce the difficulties of maintaining the identity, keeping the original 3D motion, and avoiding shape distortions, we disentangle the StyleGAN2 latent vectors of human face video frames to decouple the appearance, shape, expression, and motion from identity. An edit encoding module is used to map a sequence of image frames to continuous latent codes with 3D parametric control and is trained in a self-supervised manner with identity loss and triple shape losses. Our model supports propagation of edits in various forms: I. direct appearance editing on a specific keyframe, II. implicit editing of face shape via a given reference image, and III. existing latent-based semantic edits. Experiments show that our method works well for various forms of videos in the wild and outperforms an animation-based approach and the recent deep generative techniques.

Deep Scene Flow Learning: From 2D Images to 3D Point Clouds.

Scene flow describes the 3D motion in a scene. It can be modeled as a single task or as a composite of the auxiliary tasks of depth, camera motion, and optical flow estimation. Deep learning's emergence in recent years has broadened the horizons for new methodologies in estimating these tasks, either as separate tasks or as joint tasks to reconstruct the scene flow. The sequence of images that are either synthesized or captured by a camera is used as input for these methods, which face the challenge of dealing with various situations in images to provide the most accurate motion, such as image quality. Nowadays, images have been superseded by point clouds, which provide 3D information, thereby expediting and enhancing the estimated motion. In this paper, we dig deeply into scene flow estimation in the deep learning era. We provide a comprehensive overview of the important topics regarding both image-based and point-cloud-based methods. In addition, we cover the methodologies for each category, highlighting the network architecture. Furthermore, we provide a comparison between these methods in terms of performance and efficiency. Finally, we conclude this survey with insights and discussions on the open issues and future research directions.

Evaluation of the Vmaxpro sensor for assessing movement velocity and load-velocity variables: accuracy and implications for practical use.

We investigated the ecological validity of an inertial measurement unit (IMU) (Vmaxpro) to assess the movement velocity (MV) during a 1-repetition maximum (1RM) test and for the prediction of load-velocity (L-V) variables, as well as the ecological intra- day and inter-day reliability during free-weight bench press (BP) and squat (SQ). Furthermore, we provide recommendations for the practical use of the sensor. Twenty-three strength-trained men completed an incremental 1RM test, whereas seventeen men further participated in another 3 sessions consisting of 3 repetitions with 4 different loads (30, 50, 70 and 90% of 1RM) to assess validity and intra- and inter-day reliability, respectively. The MV was assessed using the Vmaxpro and a 3D motion capture system (MoCap). L-V variables and the 1RM were calculated based on submaximal velocities. The Vmaxpro showed high validity during the 1RM test for BP (r = 0.935) and SQ (r = 0.900), but with decreasing validity at lower MVs. The L-V variables and the 1RM demonstrated high validity for BP (r = 0.808-0.942) and SQ (r = 0.615-0.741) with a systematic overestimation. Coefficients of variance for intra- and inter-day reliability ranged from 2.4% to 9.7% and from 3.2% to 8.6% for BP and SQ, respectively. The Vmaxpro appears valid at high and moderately valid at low MVs. Depending on the required degree of accuracy, the sensor may be sufficient for the prediction of L-V variables and the 1RM. Our data indicate the sensor to be suitable for monitoring changes in MVs within and between training sessions.

Synthetic IMU Datasets and Protocols Can Simplify Fall Detection Experiments and Optimize Sensor Configuration.

Falls represent a significant cause of injury among the elderly population. Extensive research has been devoted to the utilization of wearable IMU sensors in conjunction with machine learning techniques for fall detection. To address the challenge of acquiring costly training data, this paper presents a novel method that generates a substantial volume of synthetic IMU data with minimal actual fall experiments. First, unmarked 3D motion capture technology is employed to reconstruct human movements. Subsequently, utilizing the biomechanical simulation platform Opensim and forward kinematic methods, an ample amount of training data from various body segments can be custom generated. Synthetic IMU data was then used to train a machine learning model, achieving testing accuracies of 91.99% and 86.62% on two distinct datasets of actual fall-related IMU data. Building upon the simulation framework, this paper further optimized the single IMU attachment position and multiple IMU combinations on fall detection. The proposed method simplifies fall detection data acquisition experiments, provides novel venue for generating low cost synthetic data in scenario where acquiring data for machine learning is challenging and paves the way for customizing machine learning configurations.

Incorporating variational auto-encoder networks for text-driven generation of 3D motion human body

Incorporating variational auto-encoder networks for text-driven generation of 3D motion human body

Human Pose Estimation for Clinical Analysis of Gait Pathologies.

Gait analysis serves as a critical diagnostic tool for identifying neurologic and musculoskeletal damage. Traditional manual analysis of motion data, however, is labor-intensive and heavily reliant on the expertise and judgment of the therapist. This study introduces a binary classification method for the quantitative assessment of gait impairments, specifically focusing on Duchenne muscular dystrophy (DMD), a prevalent and fatal neuromuscular genetic disorder. The research compares spatiotemporal and sagittal kinematic gait features derived from 2D and 3D human pose estimation trajectories against concurrently recorded 3D motion capture (MoCap) data from healthy children. The proposed model leverages a novel benchmark dataset, collected from YouTube and publicly available datasets of their typically developed peers, to extract time-distance variables (e.g. speed, step length, stride time, and cadence) and sagittal joint angles of the lower extremity (e.g. hip, knee, and knee flexion angles). Machine learning and deep learning techniques are employed to discern patterns that can identify children exhibiting DMD gait disturbances. While the current model is capable of distinguishing between healthy subjects and those with DMD, it does not specifically differentiate between DMD patients and patients with other gait impairments. Experimental results validate the efficacy of our cost-effective method, which relies on recorded RGB video, in detecting gait abnormalities, achieving a prediction accuracy of 96.2% for Support Vector Machine (SVM) and 97% for the deep network.

Rapid-Motion-Track: Markerless tracking of fast human motion with deep learning

Human movement patterns reflect central nervous system function. Small deficits in repetitive fast movements, such as slightly slowed finger-tapping or mildly irregular rhythm of stepping, are often an early sign of a neurological disorder. Accessible tools that precisely measure the individual components of fast movements would thus enhance disease detection, monitoring and research. Deep learning-based computer vision methods applied to digital video-recordings hold promise but current state-of-the-art tools, including DeepLabCut (DLC) and other advanced models, fail to accurately track the fastest range of human movements, primarily due to image blur. To solve this, we developed a new end-to-end, Rapid-Motion-Track (RMT) computer vision tool. This study aimed to evaluate the accuracy of RMT compared to DLC and other advanced computer vision tools. 220 finger-tapping tests were performed at frequencies between 0.5Hz and 6Hz and recorded simultaneously with a standard 30 frames/sec 2D laptop camera and a high-speed 250 frames/sec 3D motion tracking system (ground-truth). Bland-Altman plots and paired Welch's t-test were used to quantify the validity of movement features extracted by computer vision methods with the ground-truth. The movement features extracted by RMT (e.g. frequency, speed, variance) exhibited high concurrent validity across all tapping-frequencies. RMT outperformed other computer vision methods for very fast movements >4Hz. RMT also robustly tracked other fast motions including sit-to-stand, head-turning, foot-tapping, and leg agility. This new tool provides an accurate method to precisely and automatically measure even the fastest and finest human movements. It holds potential of wide reach as digital cameras are ubiquitous in homes, clinics and research centres.

An investigation of methods to improve the accuracy of classical dance steps based on machine vision recognition technology

Abstract Methods that use motion capture data to analyze human behavior are highly interpretable and offer significant advantages in vision-based dance step analysis. In this study, we propose a fusion feature extraction method that describes the rotational information embedded in the skeleton and combines the advantages of single-skeleton feature extraction and Li group feature extraction methods for recognition. The feasibility and validity of the proposed model are verified by conducting recognition experiments on BVH data of classical dance steps performed by subjects. In the ablation experiments, the average accuracy of the classical dance step recognition model based on fused features for step movement recognition is improved by 3.01% and 1.86% compared with the neural network model and the Lie group network model, respectively. It has been proven that adding rotation information to skeletal features can effectively differentiate 3D motion trajectories in similar dance steps. Furthermore, the dance movement trajectories derived from this model are very clear and can be utilized to direct the correct joint point positions throughout the movement. The recognition accuracy of the fusion feature extraction-based recognition method for all seven classical dance basic foot positions is greater than 90%, which reflects the accuracy of the proposed machine vision model in recognizing classical dance steps.

Bayesian tensor-on-tensor regression with efficient computation.

We propose a Bayesian tensor-on-tensor regression approach to predict a multidimensional array (tensor) of arbitrary dimensions from another tensor of arbitrary dimensions, building upon the Tucker decomposition of the regression coefficient tensor. Traditional tensor regression methods making use of the Tucker decomposition either assume the dimension of the core tensor to be known or estimate it via cross-validation or some model selection criteria. However, no existing method can simultaneously estimate the model dimension (the dimension of the core tensor) and other model parameters. To fill this gap, we develop an efficient Markov Chain Monte Carlo (MCMC) algorithm to estimate both the model dimension and parameters for posterior inference. Besides the MCMC sampler, we also develop an ultra-fast optimization-based computing algorithm wherein the maximum a posteriori estimators for parameters are computed, and the model dimension is optimized via a simulated annealing algorithm. The proposed Bayesian framework provides a natural way for uncertainty quantification. Through extensive simulation studies, we evaluate the proposed Bayesian tensor-on-tensor regression model and show its superior performance compared to alternative methods. We also demonstrate its practical effectiveness by applying it to two real-world datasets, including facial imaging data and 3D motion data.

Preparation of 4D Hydrogel with PET-RAFT and Orthogonal Photo-reaction

We conducted an experiment to compare the predicted outcomes with the practical results of synthesizing four-dimensional (4D) hydrogels and observing 4D motion. We employed a computer simulator and utilized PET-RAFT...

Robust Data Association Against Detection Deficiency for Semantic SLAM

Robust and accurate object association is essential for precise 3D object landmark inference in semantic Simultaneous Localization and Mapping (SLAM), and yet remains challenging due to the detection deficiency caused by high miss detection rate, false alarm, occlusion and limited field-of-view, etc. The 2D location of an object is a crucial complementary cue to the appearance feature, especially in the case of associating objects across frames under large viewpoint changes. However, motion model or trajectory pattern based methods struggle to infer object motion reliably with a moving camera. In this paper, by exploiting the local projective warping consistency, a local homography based 2D motion inference method is proposed to sequentially estimate the object location along with uncertainty. By integrating the deep appearance feature and semantic information, an object association method, named HOA, which is robust to detection deficiency is proposed. Experimental evaluations suggest that the proposed motion prediction method is capable of maintaining a low cumulative error over a long duration, which enhances the object association performance in both accuracy and robustness. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work aims to consistently associate 2D detection boxes corresponding to the same 3D object across images. In tasks of landmark-based navigation, collision avoidance, grasping and manipulation, objects in the task space are commonly simplified into 3D enveloping surfaces (e.g. cuboid or ellipsoid) by using 2D object detection boxes from multiple image views, and accurate data association is a prerequisite for precise enveloping surface reconstruction. This problem remains challenging considering the imperfect object detections, the appearance similarity of objects and the unpredictable trajectory of the moving camera. This work proposes a long-term reliable 2D location prediction algorithm that is capable of handling the complex motion of the target. Along with the appearance feature extracted by a retrain-free deep learning based model, this work proposes an object association method that can simultaneously deal with multiple objects with unknown object categories under the moving camera scenario.

Gait alterations in patients with adult spinal deformity

Adult spinal deformity patients (ASD) experience altered spinal alignment affecting spatiotemporal parameters and joint kinematics. Differences in spinal deformity between patients with symptomatic idiopathic scoliosis (ID-ASD) and patients with "de novo" scoliosis (DN-ASD) may affect gait characteristics differently. This study aims to compare gait characteristics between ID-ASD, DN-ASD, and asymptomatic healthy matched controls. In this observational case-control study, ID-ASD (n=24) and DN-ASD (n=26) patients visiting the out-patient spine clinic and scheduled for long-segment spinal fusion were included. Patients were matched, based on age, gender, leg length and BMI, with asymptomatic healthy controls. Gait was measured at comfortable walking speed on an instrumented treadmill with 3D motion capture system. Trunk, pelvic and lower extremities range of motion (ROM) and spatiotemporal parameters (SPT) are presented as median (first and thirds quartile). Independent t-test or Mann-Whitney U test was used to compare ID-ASD, DN-ASD and controls. Statistical Parametric Mapping (independent t-test) was used to compare 3D joint kinematics. DN-ASD patients walk with increased anterior trunk tilt during the whole gait cycle compared with ID-ASD patients and controls. ID-ASD walk with decreased trunk lateroflexion compared with DN-ASD and controls. DN-ASD showed decreased pelvic obliquity and -rotation, increased knee flexion, and decreased ankle plantar flexion. ID-ASD and DN-ASD displayed decreased trunk, pelvic and lower extremity ROM compared with controls, but increased pelvic tilt ROM. ID-ASD patients walked with comparable SPT to controls, whereas DN-ASD patients walked significantly slower with corresponding changes in SPT and wider steps. DN-ASD patients exhibit distinct alterations in SPT and kinematic gait characteristics compared with ID-ASD and controls. These alterations seem to be predominantly influenced by sagittal spinal malalignment and kinematic findings in ASD patients should not be generalized as such, but always be interpreted with consideration for the nature of the ASD.

RETRACTED ARTICLE:3D motion trajectory prediction based on optical image remote sensing data in sports training simulation

RETRACTED ARTICLE:3D motion trajectory prediction based on optical image remote sensing data in sports training simulation