Human Motion Data Research Articles

Mobile Phone Intelligent Recognition of Human Activity Based on Deep Neural Network

Abstract. Because of the rapid development of smartphone sensor technology and the continuous progress of machine learning algorithms, it has become possible to use smartphones for human activity recognition. Sensors such as accelerometers, gyroscopes, and magnetometers built into smartphones are able to collect human motion data in different activities, which provides a rich data source for activity recognition through mobile phones. In this paper, we change the activation function of the hidden layer of the neural network to observe the effect of this variable on the mobile phone to recognize human activities. Taking Rectified Linear Unit (ReLU)function and Sigmoid function as examples, it is found that the neural network using the ReLU function shows higher accuracy (0. 9518 and 0. 9332) and lower loss value in the early stage and middle to late stage of training. In addition, This paper finds that training for 100 epochs takes significantly less time than the model with the Sigmoid activation function (47. 041 seconds vs 179. 022 seconds). The neural network system built by the relu function has faster operation speed and better performance. This study shows that different hidden layer functions have great influence on the quality of the same neural network, and selecting the best hidden layer function will be an important point for the success of the study.

Enhancing intention prediction and interpretability in service robots with LLM and KG

The rapid advancement of artificial intelligence has significantly expanded the role of service robots in everyday life. This expansion necessitates the accurate recognition and prediction of human intentions to provide timely and appropriate services. However, existing methods often struggle to perform effectively in complex and unstructured environments. To address this challenge, we propose the Large language model and Knowledge graph based Intention Recognition Framework (LKIRF), which combines large language model (LLM) with knowledge graphs (KG) to enhance the intention recognition capabilities of service robots. Our approach constructs an offline KG from human motion and environmental data and builds an online reasoning graph through real-time interaction, utilizing LLM for interpretation. Experimental results indicate that compared to traditional methods, LKIRF not only improves prediction accuracy across various scenarios but also enhances the transparency and interpretability of the intention reasoning process.

A network evaluation of human and animal movement data across multiple swine farm systems in North America

INTRODUCTIONThe U.S. swine industry is vulnerable to the rapid spread of disease due to systemic structural issues. While animal movement networks are used to identify disease spread risks and design response plans, human movement between farms were rarely accounted for. Human movements, when integrated with animal movement models, create a different, more inclusive, and accurate network structure when compared to animal movements alone. METHODSOne year of propriety farm visit data was analyzed and consisted of anonymized property IDs, location, and user/truck IDs, along with visit dates, property, vehicle, and entry types from three swine management companies. A static directed network was created using the igraph package in R for all movements, with separate sub-networks for each entry type (animal, human, and subsets of vehicle types). Network statistics for each sub-network were compared. RESULTSThe full network included 455 properties, 11 property types, 9 vehicle types, 12 entry types, and 320001 edges (trips between properties). The longest path length was 10 in the animal movement network but decreased to 5 for the full and human movement network, while the average path length decreased from 3.2 to 2.2. Edge density increased from 0.03 to 0.09 for the human network and 0.1 for the full network. For all network properties examined, the full and human movement networks demonstrated higher connectivity than the animal network. A heavy right skew in the degree distributions indicates a 'hub' structure (scale-free-like network) and the shorter path lengths indicates a small-world network topology. DISCUSSIONThe full network is very well connected, more so than expected based on animal movement alone. Hubs may indicate points of disease susceptibility and 'super-spreader' properties. The high connectivity shows that swine farm networks may be more susceptible to spready of an introduced disease than expected from previous analyses. CONCLUSIONSMonitoring human, as well as animal movement, provides for a more complete and accurate understanding of swine farm biosecurity risks.

An Efficient Attentive-GRU Structure for Uncertainty Modelling of Crowdsourced Human Trajectories Under Building-obscured Urban Scenes

Abstract. Uncertainty modelling is regarded as one of the core components in the field of human mobility analysis and urban navigation, that can affect the performance of human behaviour modelling and location information acquisition. Existing uncertainty modelling algorithms towards the human movement trajectory are subjected to random and highly dynamic human motion characteristics and sampling and observation errors of Global Navigation Satellite System (GNSS) signals caused by the occlusion of buildings in urban scenes, which lead to the insufficient spatiotemporal correlation and poor accuracy of uncertainty modelling. To fill in this gap, this paper proposes an efficient attentive-GRU structure for uncertainty modelling of crowdsourced human trajectories under building-obscured urban scenes, that takes into account both temporal correlation and spatial correlation of human-originated GNSS trajectories and related motion features. A period of human motion data is modelled instead of only one or adjacent location points to avoid interference factors caused by the obstruction of urban buildings, and time-varying measurement and sampling errors are further estimated and combined with comprehensive human motion features to improve the accuracy of final uncertainty modelling. Comprehensive experiments indicate that compared with existing uncertainty modelling methods including physical models and deep-learning models, the proposed attentive-GRU structure realizes much better performance under different accuracy indexes.

Enhancing Digital Twins with Human Movement Data: A Comparative Study of Lidar-Based Tracking Methods

Digitals twins, used to represent dynamic environments, require accurate tracking of human movement to enhance their real-world application. This paper contributes to the field by systematically evaluating and comparing pre-existing tracking methods to identify strengths, weaknesses and practical applications within digital twin frameworks. The purpose of this study is to assess the efficacy of existing human movement tracking techniques for digital twins in real world environments, with the goal of improving spatial analysis and interaction within these virtual modes. We compare three approaches using indoor-mounted lidar sensors: (1) a frame-by-frame method deep learning model with convolutional neural networks (CNNs), (2) custom algorithms developed using OpenCV, and (3) the off-the-shelf lidar perception software package Percept version 1.6.3. Of these, the deep learning method performed best (F1 = 0.88), followed by Percept (F1 = 0.61), and finally the custom algorithms using OpenCV (F1 = 0.58). Each method had particular strengths and weaknesses, with OpenCV-based approaches that use frame comparison vulnerable to signal instability that is manifested as “flickering” in the dataset. Subsequent analysis of the spatial distribution of error revealed that both the custom algorithms and Percept took longer to acquire an identification, resulting in increased error near doorways. Percept software excelled in scenarios involving stationary individuals. These findings highlight the importance of selecting appropriate tracking methods for specific use. Future work will focus on model optimization, alternative data logging techniques, and innovative approaches to mitigate computational challenges, paving the way for more sophisticated and accessible spatial analysis tools. Integrating complementary sensor types and strategies, such as radar, audio levels, indoor positioning systems (IPSs), and wi-fi data, could further improve detection accuracy and validation while maintaining privacy.

Stretchable glove for accurate and robust hand pose reconstruction based on comprehensive motion data

We propose a compact wearable glove capable of estimating both the finger bone lengths and the joint angles of the wearer with a simple stretch-based sensing mechanism. The soft sensing glove is designed to easily stretch and to be one-size-fits-all, both measuring the size of the hand and estimating the finger joint motions of the thumb, index, and middle fingers. The system was calibrated and evaluated using comprehensive hand motion data that reflect the extensive range of natural human hand motions and various anatomical structures. The data were collected with a custom motion-capture setup and transformed into the joint angles through our post-processing method. The glove system is capable of reconstructing arbitrary and even unconventional hand poses with accuracy and robustness, confirmed by evaluations on the estimation of bone lengths (mean error: 2.1 mm), joint angles (mean error: 4.16°), and fingertip positions (mean 3D error: 4.02 mm), and on overall hand pose reconstructions in various applications. The proposed glove allows us to take advantage of the dexterity of the human hand with potential applications, including but not limited to teleoperation of anthropomorphic robot hands or surgical robots, virtual and augmented reality, and collection of human motion data.

Research on the Human Motion Recognition Method Based on Wearable.

The accurate analysis of human dynamic behavior is very important for overcoming the limitations of movement diversity and behavioral adaptability. In this paper, a wearable device-based human dynamic behavior recognition method is proposed. The method collects acceleration and angular velocity data through a six-axis sensor to identify information containing specific behavior characteristics in a time series. A human movement data acquisition platform, the DMP attitude solution algorithm, and the threshold algorithm are used for processing. In this experiment, ten volunteers wore wearable sensors on their bilateral forearms, upper arms, thighs, calves, and waist, and movement data for standing, walking, and jumping were collected in school corridors and laboratory environments to verify the effectiveness of this wearable human movement recognition method. The results show that the recognition accuracy for standing, walking, and jumping reaches 98.33%, 96.67%, and 94.60%, respectively, and the average recognition rate is 96.53%. Compared with similar methods, this method not only improves the recognition accuracy but also simplifies the recognition algorithm and effectively saves computing resources. This research is expected to provide a new perspective for the recognition of human dynamic behavior and promote the wider application of wearable technology in the field of daily living assistance and health management.

FLK: A filter with learned kinematics for real-time 3D human pose estimation

There is a growing interest in adopting 3D human pose estimation in safety-critical systems, from healthcare to Industry 5.0. Nevertheless, when applied in such settings, these neural networks may suffer from estimation inaccuracy. Besides imprecise or inconsistent annotations in the training dataset, the inaccuracy is caused by poor image quality, rare poses, dropped frames, or heavy occlusions in the scene. In addition, these scenarios often require the software results to have temporal constraints, such as real-time and zero- or low-latency, which make many of the filtering solutions proposed in the literature inapplicable. This paper proposes FLK, a Filter with Learned Kinematics, to refine 3D human motion data in real-time and at zero/low latency. The temporal core combines a Kalman filter and a low-pass filter, which learns the motion model through a recurrent neural network. The spatial core takes advantage of the biomechanical constraints of the human body to provide spatial coherency between keypoints. The combination of the cores allows the filter to adequately address different types of noise, from jittering to dropped frames. We test the filter on motion data from multiple datasets and seven 3D human pose estimation backbones, improving accuracy up to 140 mm with non-Gaussian noise and 53 mm with missing information.

Machine Learning-Based Human Movements Mimicking System for Animation and Virtual Reality

A trend, which can be noticed over the last several years, is the increasing interest to the machine learning methods in the animation and virtual reality (VR) industries to process the human motion data. The present abstract studies the possible contribution of human like machine learning simulations on human motions, which aim to improve animation and motion synthesis personalization. In this research, we propose a novel framework that utilizes camera-captured human motion data. This technique, which takes advantage of comes of motion data to simulate these animated characters movements in real time, will allow animators to create more creature-like animated figures with subtle actions as seen in real life. Additionally, in virtual reality, it links users’ movements with their avatars so the whole interaction is more engaging and realistic.

Human motion data expansion from arbitrary sparse sensors with shallow recurrent decoders.

Advances in deep learning and sparse sensing have emerged as powerful tools for monitoring human motion in natural environments. We develop a deep learning architecture, constructed from a shallow recurrent decoder network, that expands human motion data by mapping a limited (sparse) number of sensors to a comprehensive (dense) configuration, thereby inferring the motion of unmonitored body segments. Even with a single sensor, we reconstruct the comprehensive set of time series measurements, which are important for tracking and informing movement-related health and performance outcomes. Notably, this mapping leverages sensor time histories to inform the transformation from sparse to dense sensor configurations. We apply this mapping architecture to a variety of datasets, including controlled movement tasks, gait pattern exploration, and free-moving environments. Additionally, this mapping can be subject-specific (based on an individual's unique data for deployment at home and in the community) or group-based (where data from a large group are used to learn a general movement model and predict outcomes for unknown subjects). By expanding our datasets to unmeasured or unavailable quantities, this work can impact clinical trials, robotic/device control, and human performance by improving the accuracy and availability of digital biomarker estimates.

Modeling rehabilitation dataset to implement effective AI assistive systems

The demand for automated systems monitoring and supporting patients in their home-based recovery programs is substantial. While emerging technologies have been proposed as potential solutions to enhance at-home patient care, limited systems are in place due to their challenges in offering real-time monitoring and corrective feedback. Most proposed methodologies provide an overall measure or score for the executed movement. The proposed work involves the adaptation of an existing published dataset to cater to the needs of a system capable of remotely assisting patients, effectively acting as a virtual physical therapist able to provide corrective feedback. A dataset containing a set of three simple exercises for shoulder rehabilitation was processed. Each movement was meticulously annotated for temporal and categorical motion domains to monitor exercise execution in terms of the range of motion completeness and to evaluate compensatory movement patterns. This work carries substantial significance by offering a standardized and easily accessible model for human movement data, thus fostering the advancement of digital assistive systems designed to support home-based rehabilitation programs.

Older Adult Stair and Walking Speeds from Controlled Trials as Inputs into Simulations of Retirement Home Evacuation

This work aims to add to the current database of human movement data, specifically, 3-stair descent speeds from controlled trials of typically aging older adults (n=212, aged 60-99) with and without mobility impairments. Additionally, to explore the impacts of model input parameters, stair and walking speeds obtained from controlled trials are used to simulate first-iteration Canadian retirement home egress scenarios with Pathfinder. Analysis of descending stair speeds reveal interesting trends and key inter-population variations. Moreover, when the stair and horizontal walking speeds determined from these controlled trials are used as inputs into the illustrative egress simulations, a difference in predicted egress outcomes are seen compared to when model defaults are used. Together, the findings indicate important egress considerations for a vulnerable population.

HME-KG: A method of constructing the human motion encoding knowledge graph based on a hierarchical motion model

The diversity, infinity, and nonuniform description of human motion make it challenging for computers to understand human activities. To explore and reuse captured human motion data, this work defines a more comprehensive hierarchical theoretical model of human motion and proposes a standard human posture encoding scheme. We construct a domain knowledge graph (DKG) named the human motion encoding knowledge graph (HME-KG) based on posture codes and action labels. Community detection, similarity analysis, and centrality analysis are used to explore the potential value of motion data. This paper conducts an evaluation and visualization of HME-KG.

Integrating human-centric simulations in educational production lines: advancing ergonomics for industry 5.0 applications

This research in the Industry 5.0 field focuses on a human-centered simulation of the FAS200 SMC educational production line, utilizing Tecnomatix Process Simulate Human software for developing a virtual human environment. A key aspect of this study is the integration of inertial sensors, enhancing the accuracy and depth of ergonomic analysis. These sensors play a pivotal role in capturing precise human movement data, crucial for ergonomic assessments. Adopting a defined working methodology, the study extensively employs the RULA method to evaluate operator postures in the production line. This approach has led to significant ergonomic improvements, evidenced by a 40 percent reduction in the RULA index at each workstation. The integration of inertial sensors has been instrumental in achieving these results, providing detailed insights into human movements and interactions with the production environment. The research transcends traditional ergonomic assessments by incorporating a new human-centered approach, emphasizing the well-being of individuals working alongside machines. This approach, bolstered by the use of inertial sensors, marks a significant advancement in ergonomic studies, aligning with the principles of Industry 5.0. The findings hold substantial potential for application in industrial settings, signaling a shift towards more human-friendly and efficient industrial practices.

YJMob100K: City-scale and longitudinal dataset of anonymized human mobility trajectories

Modeling and predicting human mobility trajectories in urban areas is an essential task for various applications including transportation modeling, disaster management, and urban planning. The recent availability of large-scale human movement data collected from mobile devices has enabled the development of complex human mobility prediction models. However, human mobility prediction methods are often trained and tested on different datasets, due to the lack of open-source large-scale human mobility datasets amid privacy concerns, posing a challenge towards conducting transparent performance comparisons between methods. To this end, we created an open-source, anonymized, metropolitan scale, and longitudinal (75 days) dataset of 100,000 individuals’ human mobility trajectories, using mobile phone location data provided by Yahoo Japan Corporation (currently renamed to LY Corporation), named YJMob100K. The location pings are spatially and temporally discretized, and the metropolitan area is undisclosed to protect users’ privacy. The 90-day period is composed of 75 days of business-as-usual and 15 days during an emergency, to test human mobility predictability during both normal and anomalous situations.

3D human pose data augmentation using Generative Adversarial Networks for robotic-assisted movement quality assessment.

In the realm of human motion recognition systems, the augmentation of 3D human pose data plays a pivotal role in enriching and enhancing the quality of original datasets through the generation of synthetic data. This augmentation is vital for addressing the current research gaps in diversity and complexity, particularly when dealing with rare or complex human movements. Our study introduces a groundbreaking approach employing Generative Adversarial Networks (GANs), coupled with Support Vector Machine (SVM) and DenseNet, further enhanced by robot-assisted technology to improve the precision and efficiency of data collection. The GANs in our model are responsible for generating highly realistic and diverse 3D human motion data, while SVM aids in the effective classification of this data. DenseNet is utilized for the extraction of key features, facilitating a comprehensive and integrated approach that significantly elevates both the data augmentation process and the model's ability to process and analyze complex human movements. The experimental outcomes underscore our model's exceptional performance in motion quality assessment, showcasing a substantial improvement over traditional methods in terms of classification accuracy and data processing efficiency. These results validate the effectiveness of our integrated network model, setting a solid foundation for future advancements in the field. Our research not only introduces innovative methodologies for 3D human pose data enhancement but also provides substantial technical support for practical applications across various domains, including sports science, rehabilitation medicine, and virtual reality. By combining advanced algorithmic strategies with robotic technologies, our work addresses key challenges in data augmentation and motion quality assessment, paving the way for new research and development opportunities in these critical areas.

Device Position-Independent Human Activity Recognition with Wearable Sensors Using Deep Neural Networks

Human activity recognition (HAR) identifies people’s motions and actions in daily life. HAR research has grown with the popularity of internet-connected, wearable sensors that capture human movement data to detect activities. Recent deep learning advances have enabled more HAR research and applications using data from wearable devices. However, prior HAR research often focused on a few sensor locations on the body. Recognizing real-world activities poses challenges when device positioning is uncontrolled or initial user training data are unavailable. This research analyzes the feasibility of deep learning models for both position-dependent and position-independent HAR. We introduce an advanced residual deep learning model called Att-ResBiGRU, which excels at accurate position-dependent HAR and delivers excellent performance for position-independent HAR. We evaluate this model using three public HAR datasets: Opportunity, PAMAP2, and REALWORLD16. Comparisons are made to previously published deep learning architectures for addressing HAR challenges. The proposed Att-ResBiGRU model outperforms existing techniques in accuracy, cross-entropy loss, and F1-score across all three datasets. We assess the model using k-fold cross-validation. The Att-ResBiGRU achieves F1-scores of 86.69%, 96.23%, and 96.44% on the PAMAP2, REALWORLD16, and Opportunity datasets, surpassing state-of-the-art models across all datasets. Our experiments and analysis demonstrate the exceptional performance of the Att-ResBiGRU model for HAR applications.

Human-like acceleration and deceleration control of a robot astronaut floating in a space station

The acceleration and deceleration (AD) motions are the basic motion modes of robot astronauts moving in a space station. Controlling the locomotion of the robot astronaut is very challenging due to the strong nonlinearity of its complex multi-body dynamics in a gravity-free environment. However, after training, humans can move well in space stations by pushing the bulkhead, and the motion mechanism of humans is a good reference for robot astronauts. The contribution of this study is modeling the human AD motion in a microgravity environment and proposing a human-like control method for robot astronauts moving in space stations. Specifically, the movement and contact force data of the human body during AD motion were collected on an air-floating platform. Through human AD modeling analysis, the mechanism of human motion is discovered, and semi-sinusoidal primitives of contact forces are proposed for AD motion. Then, a dynamic guidance model of human AD motion is built to complete motion planning under contact constraints, which is used as the expected model for the AD control of robot astronauts. Benefiting from the force primitives, accurate and safe planning of human-like AD motion can be completed. The characteristics and mechanism of human AD motion have been analyzed from the perspective of optimization. Lastly, based on the proposed dynamic guidance model, the AD motion policy is mapped to the robot astronaut system via a system control method based on the equivalent mapping of dynamic responses (force, velocity and pose). Through comparative analysis with real human motion data and simulation results under different conditions, the proposed AD control method can achieve human-like motion efficiently and stably. Even when confronted with errors in the robot’s contact velocities and inertia parameters, the method can significantly reduce the motion errors while ensuring stability.

Event-Based Non-rigid Reconstruction of Low-Rank Parametrized Deformations from Contours

Visual reconstruction of fast non-rigid object deformations over time is a challenge for conventional frame-based cameras. In recent years, event cameras have gained significant attention due to their bio-inspired properties, such as high temporal resolution and high dynamic range. In this paper, we propose a novel approach for reconstructing such deformations using event measurements. Under the assumption of a static background, where all events are generated by the motion, our approach estimates the deformation of objects from events generated at the object contour in a probabilistic optimization framework. It associates events to mesh faces on the contour and maximizes the alignment of the line of sight through the event pixel with the associated face. In experiments on synthetic and real data of human body motion, we demonstrate the advantages of our method over state-of-the-art optimization and learning-based approaches for reconstructing the motion of human arms and hands. In addition, we propose an efficient event stream simulator to synthesize realistic event data for human motion.

Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints.

Reinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks. The proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent. The effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks.