Human Motion Tracking Research Articles

DOES: A Deep Learning-Based Approach to Estimate Roll and Pitch at Sea

The use of Attitude and Heading Reference Systems (AHRS) for orientation estimation is now common practice in a wide range of applications, e.g., robotics and human motion tracking, aerial vehicles and aerospace, gaming and virtual reality, indoor pedestrian navigation and maritime navigation. The integration of the high-rate measurements can provide very accurate estimates, but these can suffer from errors accumulation due to the sensors drift over longer time scales. To overcome this issue, inertial sensors are typically combined with additional sensors and techniques. As an example, camera-based solutions have drawn a large attention by the community, thanks to their low-costs and easy hardware setup; moreover, impressive results have been demonstrated in the context of Deep Learning. This work presents the preliminary results obtained by DOES, a supportive Deep Learning method specifically designed for maritime navigation, which aims at improving the roll and pitch estimations obtained by common AHRS. DOES recovers these estimations through the analysis of the frames acquired by a low-cost camera pointing the horizon at sea. The training has been performed on the novel ROPIS dataset, presented in the context of this work, acquired using the FrameWO application developed for the scope. Promising results encourage to test other network backbones and to further expand the dataset, improving the accuracy of the results and the range of applications of the method as a valid support to visual-based odometry techniques.

Human Motion Tracking Using 3D Image Features with a Long Short-Term Memory Mechanism Model-An Example of Forward Reaching.

Human motion tracking is widely applied to rehabilitation tasks, and inertial measurement unit (IMU) sensors are a well-known approach for recording motion behavior. IMU sensors can provide accurate information regarding three-dimensional (3D) human motion. However, IMU sensors must be attached to the body, which can be inconvenient or uncomfortable for users. To alleviate this issue, a visual-based tracking system from two-dimensional (2D) RGB images has been studied extensively in recent years and proven to have a suitable performance for human motion tracking. However, the 2D image system has its limitations. Specifically, human motion consists of spatial changes, and the 3D motion features predicted from the 2D images have limitations. In this study, we propose a deep learning (DL) human motion tracking technology using 3D image features with a deep bidirectional long short-term memory (DBLSTM) mechanism model. The experimental results show that, compared with the traditional 2D image system, the proposed system provides improved human motion tracking ability with RMSE in acceleration less than 0.5 (m/s2) X, Y, and Z directions. These findings suggest that the proposed model is a viable approach for future human motion tracking applications.

Extraction of Human Motion Information from Digital Video Based on 3D Poisson Equation

Based on the 3D Poisson equation, this paper extracts the features of the digital video human body action sequence. By solving the Poisson equation on the silhouette sequence, the time and space features, time and space structure features, shape features, and orientation features can be obtained. First, we use the silhouette structure features in three-dimensional space-time and the orientation features of the silhouette in three-dimensional space-time to represent the local features of the silhouette sequence and use the 3D Zernike moment feature to represent the overall features of the silhouette sequence. Secondly, we combine the Bayesian classifier and AdaBoost classifier to learn and classify the features of human action sequences, conduct experiments on the Weizmann video database, and conduct multiple experiments using the method of classifying samples and selecting partial combinations for training. Then, using the recognition algorithm of motion capture, after the above process, the three-dimensional model is obtained and matched with the model in the three-dimensional model database, the sequence with the smallest distance is calculated, and the corresponding skeleton is outputted as the results of action capture. During the experiment, the human motion tracking method based on the university matching kernel (EMK) image kernel descriptor was used; that is, the scale invariant operator was used to count the characteristics of multiple training images, and finally, the high-dimensional feature space was mapped into the low-dimensional to obtain the feature space approximating the Gaussian kernel. Based on the above analysis, the main user has prior knowledge of the network environment. The experimental results show that the method in this paper can effectively extract the characteristics of human body movements and has a good classification effect for bending, one-foot jumping, vertical jumping, waving, and other movements. Due to the linear separability of the data in the kernel space, fast linear interpolation regression is performed on the features in the feature space, which significantly improves the robustness and accuracy of the estimation of the human motion pose in the image sequence.

Real-Time Tracking of Human Neck Postures and Movements.

Improper neck postures and movements are the major causes of human neck-related musculoskeletal disorders. To monitor, quantify, analyze, and detect the movements, remote and non-invasive based methods are being developed for prevention and rehabilitation. The purpose of this research is to provide a digital platform for analyzing the impact of human neck movements on the neck musculoskeletal system. The secondary objective is to design a rehabilitation monitoring system that brings accountability in the treatment prescribed, which is shown in the use-case model. To record neck movements effectively, a Smart Neckband integrated with the Inertial Measurement Unit (IMU) was designed. The initial task was to find a suitable position to locate the sensors embedded in the Smart Neckband. IMU-based real-world kinematic data were captured from eight research subjects and were used to extract kinetic data from the OpenSim simulation platform. A Random Forest algorithm was trained using the kinetic data to predict the neck movements. The results obtained correlated with the novel idea proposed in this paper of using the hyoid muscles to accurately detect neck postures and movements. The innovative approach of integrating kinematic data and kinetic data for analyzing neck postures and movements has been successfully demonstrated through the efficient application in a rehabilitation use case with about 95% accuracy. This research study presents a robust digital platform for the integration of kinematic and kinetic data that has enabled the design of a context-aware neckband for the support in the treatment of neck musculoskeletal disorders.

A Review: Deep Learning for 3D Reconstruction of Human Motion Detection

3D reconstruction of human motion is an important research topic in VR/AR content creation, virtual fitting, human-computer interaction and other fields. Deep learning theory has made important achievements in human motion detection, recognition, tracking and other aspects, and human motion detection and recognition is an important link in 3D reconstruction. In this paper, the deep learning algorithms in recent years, mainly used for human motion detection and recognition, are reviewed, and the existing methods are divided into three types: CNN-based, RNN-based and GNN-based. At the same time, the main stream data sets and frameworks adopted in the references are summarized. The content of this paper provides some references for the research of 3D reconstruction of human motion.

The Reliability of the Microsoft Kinect and Ambulatory Sensor-Based Motion Tracking Devices to Measure Shoulder Range-of-Motion: A Systematic Review and Meta-Analysis.

Advancements in motion sensing technology can potentially allow clinicians to make more accurate range-of-motion (ROM) measurements and informed decisions regarding patient management. The aim of this study was to systematically review and appraise the literature on the reliability of the Kinect, inertial sensors, smartphone applications and digital inclinometers/goniometers to measure shoulder ROM. Eleven databases were screened (MEDLINE, EMBASE, EMCARE, CINAHL, SPORTSDiscus, Compendex, IEEE Xplore, Web of Science, Proquest Science and Technology, Scopus, and PubMed). The methodological quality of the studies was assessed using the consensus-based standards for the selection of health Measurement Instruments (COSMIN) checklist. Reliability assessment used intra-class correlation coefficients (ICCs) and the criteria from Swinkels et al. (2005). Thirty-two studies were included. A total of 24 studies scored “adequate” and 2 scored “very good” for the reliability standards. Only one study scored “very good” and just over half of the studies (18/32) scored “adequate” for the measurement error standards. Good intra-rater reliability (ICC > 0.85) and inter-rater reliability (ICC > 0.80) was demonstrated with the Kinect, smartphone applications and digital inclinometers. Overall, the Kinect and ambulatory sensor-based human motion tracking devices demonstrate moderate–good levels of intra- and inter-rater reliability to measure shoulder ROM. Future reliability studies should focus on improving study design with larger sample sizes and recommended time intervals between repeated measurements.

Human Motion Tracking Algorithm Based on Image Segmentation Algorithm and Kinect Depth Information

Human motion recognition has an important application value in scenarios such as intelligent monitoring and advanced human-computer interaction, and it is an important research direction in the field of computer vision. Traditional human motion recognition algorithms based on two-dimensional cameras are susceptible to changes in light intensity and texture. The advent of depth sensors, especially the Kinect series with good performance and low price released by Microsoft, enables extensive research based on depth information. However, to a large extent, the depth information has not overcome these problems based on two-dimensional images. This article introduces the research background and significance of human motion recognition technology based on depth information, introduces in detail the research methods of human motion recognition algorithms based on depth information at home and abroad, and analyzes their advantages and disadvantages. The public dataset is introduced. Then, based on the depth information, a method of human motion recognition is proposed and optimized. A moving human body image segmentation method based on an improved two-dimensional Otsu method is proposed to solve the problem of inaccurate and slow segmentation of moving human body images using the two-dimensional Otsu method. In the process of constructing the threshold recognition function, this algorithm not only uses the cohesion of the pixels within the class but also considers the maximum variance between the target class and the background class. Then, the quantum particle swarm algorithm is used to find the optimal threshold solution of the threshold recognition function. Finally, the optimal solution is used to achieve accurate and fast image segmentation, which increases the accuracy of human body motion tracking by more than 30%.

A Super‐Stretchable and Highly Sensitive Carbon Nanotube Capacitive Strain Sensor for Wearable Applications and Soft Robotics

AbstractThe study of flexible and stretchable strain sensors is growing rapidly owing to the demands for human motion detection, human–machine interaction, and soft robotics. However, super‐stretchable and highly sensitive strain sensors with high linearity and low hysteresis are especially lacking, which therefore limits the use of soft strain sensors in varied practical applications. The stretchability and sensitivity of the capacitive strain sensor are constrained by the material characteristics and structure of parallel plate capacitor (theoretical gauge factor [GF] is 1). To address these limitations, a super‐stretchable and highly sensitive capacitive strain sensor composed of two strips of wrinkled carbon nanotubes‐based electrodes separated by a tape dielectric, is presented. By integrating nanomaterials and wrinkled film structure, this device achieves a GF of 2.07 at 300% strain with excellent linearity and negligible hysteresis. This is the first type of capacitive strain sensors that can achieve super‐stretchability and sensitivity simultaneously. Additionally, the sensor has a fast signal response time of ≈80 ms, and good mechanical durability during 1000 stretching and releasing cycles. The authors demonstrate the use of this sensor as a versatile wearable device for human motion tracking, and as a smart real‐time monitoring device for soft pneumatic robots.

Flexible triboelectric nanogenerator for human motion tracking and gesture recognition

Nowadays human motion tracking and gesture recognition have gained increasing attention as they allow humans to control machines through natural and intuitive behaviors by recognizing meaningful expressions. Here, we propose a stretchable, high durability and deformable triboelectric nanogenerator (F-TENG) on the base of silica gel, which can be used to not only collect mechanical energy from our ambient environment, but also monitor human motion and behaviors through the highly sensitive response to the complex states of human motion, such as stretching, folding, extrusion and hitting. The clutter in the signals can be effectively collected by the integrated filtering circuit, thus the function of real-time statistics of human motion and behaviors can be recorded and interpreted. This work promotes the broad application prospects of TENGs in artificial intelligence, sporting monitoring, and big data collection.

On the Problem of Synchronization of Virtual and Real Movements for Virtual Reality Systems

An immersive experience in modern virtual reality systems requires adjusting of various parameters. This report focuses on the problem of virtual representation of real movements of objects in space. Separately, a special class of problems is highlighted: synchronization of visual and dynamic imitation for virtual reality systems and motion platforms. Modern mass-produced virtual reality headsets do not allow effective tracking of human movements on a moving base using standard software. The element base and algorithms necessary for the successful solution of this problem are presented.

RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters

Inertial-sensor-based attitude estimation is a crucial technology in various applications, from human motion tracking to autonomous aerial and ground vehicles. Application scenarios differ in characteristics of the performed motion, presence of disturbances, and environmental conditions. Since state-of-the-art attitude estimators do not generalize well over these characteristics, their parameters must be tuned for the individual motion characteristics and circumstances. We propose RIANN, a ready-to-use, neural network-based, parameter-free, real-time-capable inertial attitude estimator, which generalizes well across different motion dynamics, environments, and sampling rates, without the need for application-specific adaptations. We gather six publicly available datasets of which we exploit two datasets for the method development and the training, and we use four datasets for evaluation of the trained estimator in three different test scenarios with varying practical relevance. Results show that RIANN outperforms state-of-the-art attitude estimation filters in the sense that it generalizes much better across a variety of motions and conditions in different applications, with different sensor hardware and different sampling frequencies. This is true even if the filters are tuned on each individual test dataset, whereas RIANN was trained on completely separate data and has never seen any of these test datasets. RIANN can be applied directly without adaptations or training and is therefore expected to enable plug-and-play solutions in numerous applications, especially when accuracy is crucial but no ground-truth data is available for tuning or when motion and disturbance characteristics are uncertain. We made RIANN publicly available.

Social robot motion planning using contextual distances observed from 3D human motion tracking

The social robot motion planning involves robots operating within the workspace and alongside humans and it is thus necessary for the robots to behave socially. The current literatures make trajectory databases to learn the human trajectory, which has a limited application since only very few out of large possibilities can be recoded. Another approach to the problem is by modelling the macroscopic behaviors, which is limited by parameters that are practically hard to set. Our approach to the problem is thus to understand human motion through a set of simple and well-known primitives, which are observed from the actual human motion. In this paper, we first perform human detection and tracking in a 3D environment. We use a stationary 3D Lidar sensor for the detection and tracking of all moving humans. Our approach detects all moving people and it also solves the problem of occlusion in several cases. We further consider several research hypotheses regarding human navigation noting how much distances the humans prefer to maintain with other humans and obstacles under different common scenarios. The experimentation is done with several subjects and their behavior is used to answer the research hypotheses. A Social Robot Motion Planning algorithm is developed by using a social potential field algorithm as a base. New social forces are added to model the different behaviors displayed by humans. The motion planning algorithm makes the robot maintain the same distances as observed with the humans and perturbating the robots makes them reach the equilibrium distance again.

D-SORM: A digital solution for remote monitoring based on the attitude of wearable devices

Background and objectiveE-health is a growing research topic, especially with the expansion of the Internet of Things (IoT). Miniaturized wearable sensors are auspicious tools for biomedicine and healthcare systems. In this paper, we present D-SORM, a sensor fusion-based digital solution intended to assist clinicians and improve their diagnosis by providing objective measurements and automatic recognition. The aim is to supply an interface for remote monitoring to the medical staff. MethodsD-SORM platform estimates the wearable device attitude based on its acquired data, and visualizes it in real-time using a graphical user interface (GUI). It also integrates two modules which serve two different medical applications. The first one is arm tele-rehabilitation, where sessions are done online. The practitioner gives the instructions while wearing the device, and the patient has to reproduce the gestures. A processing unit is dedicated to compute statistical features and calculate the success rate. The second one is human motion tracking for elderly care. A novel machine learning architecture is proposed, based on feature fusion, to predict the activities of daily living. ResultsThe rehabilitation mechanism was tested under supervised conditions, by performing a set of movements. D-SORM provides extra information and objective measurements, thus facilitates the diagnosis of clinicians. The human activity recognition is also validated using a public dataset. With D-SORM, an efficiency ranging from 97.7% to 99.65% is ensured under unsupervised conditions. ConclusionsThe proposed design constitutes a digital clinical tool for medical teams allowing remote health monitoring. It overcomes geographical barriers while providing faster and highly accurate assessment.

Learning-based simulation and modeling of unorganized chaining behavior using data generated from 3D human motion tracking

AbstractThe paper models the unorganized chaining behavior, where humans need to walk in a chain due to a constrained environment. Detection and tracking are done using a 3D LiDAR, which has the challenges of environmental noises, uncontrolled environment, and occlusions. The Kalman filter is used for tracking. The trajectories are analyzed and used to train a behavioral model. The modeling has applications in socialistic robot motion planning and simulations. Based on the results, we conclude that the trajectory prediction by our approach is more socialistic and has a lesser error when compared to the artificial potential field method.

Evaluation of the Intel RealSense T265 for tracking natural human head motion

Accurate and robust tracking of natural human head motion in natural environments is important for a number of applications including virtual and augmented reality, clinical diagnostics, as well as basic scientific research. IMU provide a versatile solution for recording inertial data including linear acceleration and angular velocity, but reconstructing head position is difficult or impossible. This problem can be solved by incorporating visual data using a technique known as visual-inertial simultaneous localization and mapping (VI-SLAM). A recently released commercial solution, the Intel RealSense T265, uses a proprietary VI-SLAM algorithm to estimate linear and angular position and velocity, but the performance of this device for tracking of natural human head motion in natural environments has not yet been comprehensively evaluated against gold-standard methods. In this study, we used a wide range of metrics to evaluate the performance of the T265 with different walking speeds in different environments, both indoor and outdoor, against two gold-standard methods, an optical tracking system and a so-called perambulator. Overall, we find that performance of the T265 relative to these gold-standard methods is most accurate for slow to normal walking speeds in small- to medium-sized environments. The suitability of this device for future scientific studies depends on the application; data presented here can be useful in making that determination.

Validity of an Artificial Intelligence-Assisted Motion-Analysis System Using a Smartphone for Evaluating Weight-Bearing Activities in Individuals with Patellofemoral Pain Syndrome

Background An artificial intelligence-assisted motion-analysis system without markers, OpenPose, is used to calculate joint angles in sports and medical analyses. Measuring the angles of the hip and knee joint in the frontal plane during the performance of weight-bearing activities is valuable in patients with patellofemoral pain syndrome (PFPS). Purpose The purpose of this study was to assess the validity of OpenPose using a pre-trained human motion-tracking algorithm for measuring the angles of the hip and knee joint in the frontal plane during standing hip abduction, semi-squat movements, and forward step-down movements compared with marker-based three-dimensional motion analysis. Study design Cross-sectional study Methods Eight individuals with PFPS participated in the current study. To investigate the validity of OpenPose, the angles of the hip and knee in the frontal plane were measured simultaneously with a smartphone camera using the OpenPose library and Vicon as the gold-standard motion-analysis system while performing three weight-bearing activities. Pearson and Spearman correlation analysis was used to assess the validity of the OpenPose-based motion-analysis system. Results Correlation coefficients ranged from 0.04 to 0.61 on the more symptomatic side and from 0.02 to 0.88 on the less symptomatic side for the three weight-bearing activities. When performing standing hip abduction and step-down movements, the validity of the measurements of hip abduction was fair or good. When performing semi-squat movements, the validity of the knee abduction measurements was fair. Conclusions The OpenPose-based motion-analysis system can provide fair or good level of validity of measurements of frontal hip and knee angles during weight-bearing activities of individuals with PFPS in real environments and for remote rehabilitation.

Optimization and Simulation of Virtual Experiment System of Human Sports Science Based on VR

Virtual reality technology is an emerging technology developed on the basis of information technology. It is widely used in military, medical, mining, entertainment, and other fields. Therefore, many countries have been vigorously conducting research in recent years. As one of the important components of the virtual reality system, the three-dimensional human motion tracking system is of great significance to the research of practical virtual reality systems. It introduces the measurement principle of the spatial three-dimensional coordinate dynamic measurement device and discusses in detail the ultrasonic transmission, reception, amplification, filtering, comparison, shaping circuit, and single-chip interface circuit. This paper introduces the working principle and characteristics of the virtual experiment system and gives the structure diagram, hardware schematic diagram, and software flow diagram of the system. We mainly study the method of tracking human motion by measuring the three-dimensional coordinates of the space point, which lays a good foundation for the research of the actual three-dimensional motion tracking system. At the same time, the three-dimensional human body modeling is discussed, and the interactive movement policy of the human arm is briefly introduced. It has a certain effect on the actual virtual reality human-computer interaction system.

Metrological Evaluation of Human-Robot Collaborative Environments Based on Optical Motion Capture Systems.

In the context of human–robot collaborative shared environments, there has been an increase in the use of optical motion capture (OMC) systems for human motion tracking. The accuracy and precision of OMC technology need to be assessed in order to ensure safe human–robot interactions, but the accuracy specifications provided by manufacturers are easily influenced by various factors affecting the measurements. This article describes a new methodology for the metrological evaluation of a human–robot collaborative environment based on optical motion capture (OMC) systems. Inspired by the ASTM E3064 test guide, and taking advantage of an existing industrial robot in the production cell, the system is evaluated for mean error, error spread, and repeatability. A detailed statistical study of the error distribution across the capture area is carried out, supported by a Mann–Whitney U-test for median comparisons. Based on the results, optimal capture areas for the use of the capture system are suggested. The results of the proposed method show that the metrological characteristics obtained are compatible and comparable in quality to other methods that do not require the intervention of an industrial robot.

Tracking rower motion without on-body sensors using an instrumented machine and an artificial neural network

Human motion tracking is an active field of research driven by its diverse applications in areas such as health care, daily activity recognition, sports, etc. In sports applications, tracking rowing motion is performed to meet several goals, including prevention of injuries, improvement of performance or provision of virtual coaching. Different established approaches rely on on-body sensors to capture rowing motion. While on-body sensors are effective and straight forward to implement, they can disturb the athlete and negatively impact the training. In this paper, an approach is presented to track rowing motion without body-worn sensors or cameras. Instead, sensors were attached to an indoor rowing machine that tracked the motion of its sliding seat, lever handle and the force exercised on the seat. In particular, the motion was tracked by means of linear and angular displacement sensors, as well as force sensors placed underneath the seat. The respective variables were fed into an Artificial Neural Network (ANN) to predict the coordinates of the rower’s shoulder, which in turn, were used to geometrically infer the angles of the shoulder, elbow, hip and thoracolumbar flexion-extension. A successful ANN architecture was iteratively designed by using the Levenberg-Marquardt algorithm and varying the number of hidden neurons in one hidden layer. A comparison between ANN-predicted and experimentally obtained shoulder coordinates from an optical motion capture system showed a mean error of less than 4 cm, which led to an angle mean error value as low as 2.01°. A rigged avatar was used to visually verify the reproduced motion. The avatar animation was well-received by experts, especially considering the shoulder adduction-abduction in the frontal plane.

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor.

High-quality and complete human motion 4D reconstruction is of great significance for immersive VR and even human operation. However, it has inevitable self-scanning constraints, and tracking under monocular settings also has strict restrictions. In this paper, we propose a human motion capture system combined with human priors and performance capture that only uses a single RGB-D sensor. To break the self-scanning constraint, we generated a complete mesh only using the front view input to initialize the geometric capture. In order to construct a correct warping field, most previous methods initialize their systems in a strict way. To maintain high fidelity while increasing the easiness of the system, we updated the model while capturing motion. Additionally, we blended in human priors in order to improve the reliability of model warping. Extensive experiments demonstrated that our method can be used more comfortably while maintaining credible geometric warping and remaining free of self-scanning constraints.