Inertial Sensor Data Research Articles

BodyFlow: An Open-Source Library for Multimodal Human Activity Recognition

Human activity recognition is a critical task for various applications across healthcare, sports, security, gaming, and other fields. This paper presents BodyFlow, a comprehensive library that seamlessly integrates human pose estimation and multiple-person estimation and tracking, along with activity recognition modules. BodyFlow enables users to effortlessly identify common activities and 2D/3D body joints from input sources such as videos, image sets, or webcams. Additionally, the library can simultaneously process inertial sensor data, offering users the flexibility to choose their preferred input, thus facilitating multimodal human activity recognition. BodyFlow incorporates state-of-the-art algorithms for 2D and 3D pose estimation and three distinct models for human activity recognition.

Quantifying Asymmetric Gait Pattern Changes Using a Hidden Markov Model Similarity Measure (HMM-SM) on Inertial Sensor Signals.

Wearable gait analysis systems using inertial sensors offer the potential for easy-to-use gait assessment in lab and free-living environments. This can enable objective long-term monitoring and decision making for individuals with gait disabilities. This study explores a novel approach that applies a hidden Markov model-based similarity measure (HMM-SM) to assess changes in gait patterns based on the gyroscope and accelerometer signals from just one or two inertial sensors. Eleven able-bodied individuals were equipped with a system which perturbed gait patterns by manipulating stance-time symmetry. Inertial sensor data were collected from various locations on the lower body to train hidden Markov models. The HMM-SM was evaluated to determine whether it corresponded to changes in gait as individuals deviated from their baseline, and whether it could provide a reliable measure of gait similarity. The HMM-SM showed consistent changes in accordance with stance-time symmetry in the following sensor configurations: pelvis, combined upper leg signals, and combined lower leg signals. Additionally, the HMM-SM demonstrated good reliability for the combined upper leg signals (ICC = 0.803) and lower leg signals (ICC = 0.795). These findings provide preliminary evidence that the HMM-SM could be useful in assessing changes in overall gait patterns. This could enable the development of compact, wearable systems for unsupervised gait assessment, without the requirement to pre-identify and measure a set of gait parameters.

Detection of aberration in human behavior using shallow neural network over smartphone inertial sensors data

AbstractThe integration of different Mobile Edge Computing (MEC) applications has significantly enhanced the realm of security and surveillance, with Human Activity Recognition (HAR) standing out as a crucial application. The diverse sensors found in smartphones have made it convenient for monitoring applications to gather and analyze data, rendering them valuable for HAR purposes. Moreover, MEC can be employed to automate surveillance, allowing intelligent monitoring of restricted areas to identify and respond to unwanted or suspicious activities. This research develops a system using motion sensors in smartphones to identify unusual human activities. People's smartphones were employed to monitor both suspicious and regular activities. Information was collected for various actions categorized as either suspicious or regular. When a person performs a certain action, the smartphone records a series of sensory data, analyse important patterns from the basic data, and then determines what the person is doing by combining information from different sensors. To prepare the data, information from different sensors was aligned to a shared timeline. In this study, we used a sliding window approach on synchronized data to feed sequences into LSTM and CNN models. These models, which include initial layers of LSTM and CNN, automatically find important patterns in the order of human activities. We combined SVM with the features extracted by the shallow Neural Network to make a mixed model that predicts suspicious activities. Lastly, we compared LSTM, CNN, and our new shallow mixed neural network using a new real‐time dataset. The mixed model of CNN and SVM achieved an accuracy of 94.43%. Additionally, the sliding window method's effectiveness was confirmed with a 4.28% improvement in accuracy.

A Smart Ski Pole for Skiing Pattern Recognition and Quantification Application.

In cross-country skiing, ski poles play a crucial role in technique, propulsion, and overall performance. The kinematic parameters of ski poles can provide valuable information about the skier's technique, which is of great significance for coaches and athletes seeking to improve their skiing performance. In this work, a new smart ski pole is proposed, which combines the uniaxial load cell and the inertial measurement unit (IMU), aiming to provide comprehensive data measurement functions more easily and to play an auxiliary role in training. The ski pole can collect data directly related to skiing technical actions, such as the skier's pole force, pole angle, inertia data, etc., and the system's design, based on wireless transmission, makes the system more convenient to provide comprehensive data acquisition functions, in order to achieve a more simple and efficient use experience. In this experiment, the characteristic data obtained from the ski poles during the Double Poling of three skiers were extracted and the sample t-test was conducted. The results showed that the three skiers had significant differences in pole force, pole angle, and pole time. Spearman correlation analysis was used to analyze the sports data of the people with good performance, and the results showed that the pole force and speed (r = 0.71) and pole support angle (r = 0.76) were significantly correlated. In addition, this study adopted the commonly used inertial sensor data for action recognition, combined with the load cell data as the input of the ski technical action recognition algorithm, and the recognition accuracy of five kinds of cross-country skiing technical actions (Diagonal Stride (DS), Double Poling (DP), Kick Double Poling (KDP), Two-stroke Glide (G2) and Five-stroke Glide (G5)) reached 99.5%, and the accuracy was significantly improved compared with similar recognition systems. Therefore, the equipment is expected to be a valuable training tool for coaches and athletes, helping them to better understand and improve their ski maneuver technique.

Removing spikes in TianQin-1 inertial sensor data using machine learning method

Abstract The spikes in the inertial sensor data have been found to impact on the retrieval of the non-gravitational signals and the evaluation of the inertial sensor performance. Removing the spikes in the inertial sensor is critical for studies of gravitational reference sensors in space-based gravitational wave detection missions and accelerometers in gravity satellite missions. Thanks to a long period of inertial sensor data without thruster spikes, we can conduct machine learning based on this data to remove spikes. In this paper, a machine learning model called bi-directional long short-term memory (Bi-LSTM) neural network was built based on the inertial sensor data of TianQin-1 (TQ-1) mission. We use the machine learning method to remove the spikes in the inertial sensor data. After removing the spikes in the inertial sensor data, acceleration noise is suppressed form 2.0×10-7ms-2Hz-1/2 to the 2.8×10-10ms-2 Hz-1/2 at 0.1 Hz, which is far better than the existing methods, including the linear interpolation, data substitution and mean value of adjacent data.

Evaluation of Patients' Levels of Walking Independence Using Inertial Sensors and Neural Networks in an Acute-Care Hospital.

This study aimed to evaluate walking independence in acute-care hospital patients using neural networks based on acceleration and angular velocity from two walking tests. Forty patients underwent the 10-m walk test and the Timed Up-and-Go test at normal speed, with or without a cane. Physiotherapists divided the patients into two groups: 24 patients who were monitored or independent while walking with a cane or without aids in the ward, and 16 patients who were not. To classify these groups, the Transformer model analyzes the left gait cycle data from eight inertial sensors. The accuracy using all the sensor data was 0.836. When sensor data from the right ankle, right wrist, and left wrist were excluded, the accuracy decreased the most. When analyzing the data from these three sensors alone, the accuracy was 0.795. Further reducing the number of sensors to only the right ankle and wrist resulted in an accuracy of 0.736. This study demonstrates the potential of a neural network-based analysis of inertial sensor data for clinically assessing a patient's level of walking independence.

OBU for Accurate Navigation through Sensor Fusion in the Framework of the EMERGE Project

With the development of advanced driver assistance systems (ADAS) and autonomous vehicles (AV), recent years have seen an increasing evolution of onboard sensors and communication interfaces capable of interacting with available infrastructures, including satellite constellations, road structures, modern and heterogeneous network systems (e.g., 5G and beyond) and even adjacent vehicles. Consequently, it is essential to develop architectures that cover data fusion (multi–sensor approach), communication, power management, and system monitoring to ensure accurate and reliable perception in several navigation scenarios. Motivated by the EMERGE project, this paper describes the definition and implementation of an On Board Unit (OBU) dedicated to the navigation process. The OBU is equipped with the Xsens MTi–630 AHRS inertial sensor, a multi–constellation/multi–frequency Global Navigation Satellite System (GNSS) receiver with the u–blox ZED–F9P module and communication interfaces that afford access to the PointPerfect augmentation service. Experimental results show that GNSS, with corrections provided by augmentation, affords centimetre accuracy, with a Time To First Fix (TTFF) of about 30 s. During the on–road tests, we also collect: the output of fusion with inertial sensor data, monitoring information that assess correct operation of the module, and the OBU power consumption, that remains under 5 W even in high–power operating mode.

Exploring raw data transformations on inertial sensor data to model user expertise when learning psychomotor skills

This paper introduces a novel approach for leveraging inertial data to discern expertise levels in motor skill execution, specifically distinguishing between experts and beginners. By implementing inertial data transformation and fusion techniques, we conduct a comprehensive analysis of motor behaviour. Our approach goes beyond conventional assessments, providing nuanced insights into the underlying patterns of movement. Additionally, we explore the potential for utilising this data-driven methodology to aid novice practitioners in enhancing their performance. The findings showcase the efficacy of this approach in accurately identifying proficiency levels and lay the groundwork for personalised interventions to support skill refinement and mastery. This research contributes to the field of motor skill assessment and intervention strategies, with broad implications for sports training, physical rehabilitation, and performance optimisation across various domains.

Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations.

Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.

Design a Deep Learning Model for Dairy Cow Abnormal Behaviour Prediction

When it comes to determining the health and well-being of cattle, it is critical to observe their behaviour. Another strong and cost-effective monitoring approach is a neural network-based monitoring system that analyses time series data from inertial sensors attached to cows. This is one of the most powerful and cost-effective monitoring systems currently available. Even though deep learning has made significant advances in pattern detection, large datasets are still required in many cases. When only a little amount of data is available, data augmentation is a very effective and low-cost pre-processing step for neural network-based systems. To study and analysis, a variety of ways for enhancing inertial sensor data based on cow behavioural data are discussed. Using convolutional neural networks, researchers are attempting to solve the difficult problem of identifying cow behaviour, which is made much more difficult by a scarcity of training data. Our proposed approaches areimproving the effectiveness of deep learning (CNN+LSTM) in the classification of cow behaviour while also lowering the total system costs associated with data collection and labelling.

Inertial Measuring System to Evaluate Gait Parameters and Dynamic Alignments for Lower-Limb Amputation Subjects.

The study aims to construct an inertial measuring system for the application of amputee subjects wearing a prosthesis. A new computation scheme to process inertial data by installing seven wireless inertial sensors on the lower limbs was implemented and validated by comparing it with an optical motion capture system. We applied this system to amputees to verify its performance for gait analysis. The gait parameters are evaluated to objectively assess the amputees' prosthesis-wearing status. The Madgwick algorithm was used in the study to correct the angular velocity deviation using acceleration data and convert it to quaternion. Further, the zero-velocity update method was applied to reconstruct patients' walking trajectories. The combination of computed walking trajectory with pelvic and lower limb joint motion enables sketching the details of motion via a stickman that helps visualize and animate the walk and gait of a test subject. Five participants with above-knee (n = 2) and below-knee (n = 3) amputations were recruited for gait analysis. Kinematic parameters were evaluated during a walking test to assess joint alignment and overall gait characteristics. Our findings support the feasibility of employing simple algorithms to achieve accurate and precise joint angle estimation and gait parameters based on wireless inertial sensor data.

Localization of solar panel cleaning robot combining vision processing and extended Kalman filter.

In this study, we introduce a method for estimating the position of a self-driving solar panel-cleaning mobile robot. This estimation relies on line counts, typically 16 cm in panel width, obtained through image processing on the panel floor, along with wheel encoder information and inertial sensor data. To achieve accurate line counts, we introduce two adjusted threshold values and allow offsets in these values based on the robot's speed. Additionally, inertial measurement unit (IMU) signals assist in determining whether a line is horizontal or vertical, depending on the robot's movement direction on the panel, utilizing the robot's heading angle and detected line angle. When the robot is positioned between lines on the panel, more precise location estimation is necessary beyond simple line counts. To tackle this challenge, we integrate the extended Kalman filter with IMU data and encoder information, significantly enhancing position estimation. This integration achieves an RMSE accuracy value of up to 0.089 m, notably at a relatively high speed of 100 mm/s. This margin of error is almost half that of the vision-based line-counting method.

Early Prediction of Poststroke Rehabilitation Outcomes Using Wearable Sensors.

Inpatient rehabilitation represents a critical setting for stroke treatment, providing intensive, targeted therapy and task-specific practice to minimize a patient's functional deficits and facilitate their reintegration into the community. However, impairment and recovery vary greatly after stroke, making it difficult to predict a patient's future outcomes or response to treatment. In this study, the authors examined the value of early-stage wearable sensor data to predict 3 functional outcomes (ambulation, independence, and risk of falling) at rehabilitation discharge. Fifty-five individuals undergoing inpatient stroke rehabilitation participated in this study. Supervised machine learning classifiers were retrospectively trained to predict discharge outcomes using data collected at hospital admission, including patient information, functional assessment scores, and inertial sensor data from the lower limbs during gait and/or balance tasks. Model performance was compared across different data combinations and was benchmarked against a traditional model trained without sensor data. For patients who were ambulatory at admission, sensor data improved the predictions of ambulation and risk of falling (with weighted F1 scores increasing by 19.6% and 23.4%, respectively) and maintained similar performance for predictions of independence, compared to a benchmark model without sensor data. The best-performing sensor-based models predicted discharge ambulation (community vs household), independence (high vs low), and risk of falling (normal vs high) with accuracies of 84.4%, 68.8%, and 65.9%, respectively. Most misclassifications occurred with admission or discharge scores near the classification boundary. For patients who were nonambulatory at admission, sensor data recorded during simple balance tasks did not offer predictive value over the benchmark models. These findings support the continued investigation of wearable sensors as an accessible, easy-to-use tool to predict the functional recovery after stroke. Accurate, early prediction of poststroke rehabilitation outcomes from wearable sensors would improve our ability to deliver personalized, effective care and discharge planning in the inpatient setting and beyond.

A Rapid Response System for Elderly Safety Monitoring Using Progressive Hierarchical Action Recognition.

The global trend of population aging presents an urgent challenge in ensuring the safety and well-being of elderly individuals, especially those living alone due to various circumstances. A promising approach to this challenge involves leveraging Human Action Recognition (HAR) by integrating data from multiple sensors. However, the field of HAR has struggled to strike a balance between accuracy and response time. While technological advancements have improved recognition accuracy, complex algorithms often come at the expense of response time. To address this issue, we introduce an innovative asynchronous detection method called Rapid Response Elderly Safety Monitoring (RESAM), which relies on progressive hierarchical action recognition and multi-sensor data fusion. Through initial analysis of inertial sensor data using Kernel Principal Component Analysis (KPCA) and multi-class classifiers, we efficiently reduce processing time and lower the false-negative rate (FNR). The inertial sensor identification serves as a pre-filter, enabling the identification of filtered abnormal signals. Decision-level data fusion is then executed, incorporating skeleton image analysis based on ResNet and the inertial sensor data from the initial step. This integration enables the accurate differentiation between normal and abnormal behaviors. The RESAM method achieves an impressive 97.4% accuracy on the UTD-MHAD database with a minimal delay of 1.22 seconds. On our internally collected database, the RESAM system attains an accuracy of 99%, ranking among the most accurate state-of-the-art methods available. These results underscore the practicality and effectiveness of our approach in meeting the critical demand for swift and precise responses in healthcare scenarios.

Early Detection of Parkinson's Disease Using Deep NeuroEnhanceNet With Smartphone Walking Recordings.

With the development of digital medical technology, ubiquitous smartphones are emerging as valuable tools for the detection of complex and elusive diseases. This paper exploits smartphone walking recording for early detection of Parkinson's disease (PD) and finds that walking recording empowered by deep learning is a valid digital biomarker for early-recognizing PD patients. Specifically, the inertial sensor data is preprocessed, including normalization, scaling, and rotation, and then the processed data is fed into the proposed deep NeuroEnhanceNet. Finally, determine the individual prediction score using the PD-prone strategy and generate the detection results. The proposed deep NeuroEnhanceNet, specifically designed for inertial sensor data, can focus on both the long-term data characteristics within a single channel and the inter-channel correlations. Our method obtains a low false negative rate of 0.053 for the early detection of PD. We further analyze and compare the effectiveness of digital biomarkers captured from the walking and resting processes for early detection of PD. All the code for this work is available at: https://github.com/heyiyia/NeuroEnhanceNet.

A Novel Fall Detection Framework Using Skip-DSCGAN Based on Inertial Sensor Data

A Novel Fall Detection Framework Using Skip-DSCGAN Based on Inertial Sensor Data

Azimuth estimation based on CNN and LSTM for geomagnetic and inertial sensors data

Although estimating the azimuth using a geomagnetic sensor is very useful, the estimation error may be very large due to the surrounding geomagnetic disturbance. We proposed a novel method for preprocessing appropriately for geomagnetic and inertial sensor data to be suitable for the proposed Artificial Neural Network model and training method for the model. As a result, the probability of azimuth estimation error within 1 degree is 96.4% with regression estimation. For classification estimation, when the azimuth estimation probability is 90% or more, the probability that the azimuth estimation error is within 1 degree is 100%.

A Yaw tracking algorithm for head movement from inertial sensors data

Monitoring head movements is important in many aspects of life from medicine and rehabilitation to sports, and VR entertainment. In this study, we used recordings from two sensors, i.e. an accelerometer and a gyroscope, to calculate the angles of movement of the gesturing person’s head. For the yaw motion, we proposed an original algorithm using only these two inertial sensors and the detected motion type obtained from a pre-trained SVM classifier. The combination of the gyroscope data and the detected motion type allowed us to calculate the yaw angle without the need for other sensors, such as a magnetometer or a video camera. To verify the accuracy of our algorithm, we used a robotic arm that simulated head gestures where the angle values were read out from the robot kinematics. The calculated yaw angles differed from the robot’s readings with a mean absolute error of approx. 1 degree and the rate of differences between these values exceeding 5 degrees was significantly below 1 percent except for one outlier at 1.12%. This level of accuracy is sufficient for many applications, such as VR systems, human-system interfaces, or rehabilitation.

ConGPS: A Smart Container Positioning System Using Inertial Sensor and Electronic Map with Infrequent GPS.

Real-time global positioning is important for container-based logistics. However, a challenge in real-time global positioning arises from the frequency of both global positioning system (GPS) calls and GPS-denied environments during transportation. This paper proposes a novel system named ConGPS that integrates both inertial sensor and electronic map data. ConGPS estimates the speed and heading direction of a moving container based on the inertial sensor data, the container trajectory, and the speed limit information provided by an electronic map. The directional information from magnetometers, coupled with map-matching algorithms, is employed to compute container trajectories and current positions. ConGPS significantly reduces the frequency of GPS calls required to maintain an accurate current position. To evaluate the accuracy of the system, 280 min of driving data, covering a distance of 360 km, are collected. The results demonstrate that ConGPS can maintain positioning accuracy within a GPS-call interval of 15 min, even if using low-cost inertial sensors in GPS-denied environments.

Investigating Optimal Smartphone Placement for Identifying Stairs Movement using Machine Learning

The identification of gait activities such as stair ascending and descending poses a significant challenge due to the proximity of data provided by the sensory pathway. Accurate identification of gait activities is crucial in conveying essential gait information to users for the recognition of human movement activities. However, gait patterns can vary significantly between individuals, making it challenging to develop a generalized algorithm for identifying incline surface gait activity. Factors such as walking speed, stride length, and body mechanics can all influence gait patterns, making it difficult to establish a consistent framework. Despite various research on gait event detection for level ground walking, the identification of gait activities on an inclined surface such as stairs, especially using smartphones as sensors, is currently lacking. The goal of this study is to investigate and develop a reliable and accurate method for detecting gait activities on an inclined surface such as stairs using smartphones as the sensing device. Specifically, this study focuses on investigating optimal placement of smartphones to extract tri- axis accelerometer data from the inertial sensors during stair movement. The inertial sensor data was collected from the smartphone at two different positions and two different orientations. The data was trained against 6 machine learning algorithms namely Decision Tree, Logistic Regression, Naive Bayes, Random Forest, Neural Networks and KNN. It was observed that, by using Decision Tree and Random Forest algorithm 100% accuracy was achieved, when smartphone was placed at the thigh during stair movement. Successful identification of stair movement activity by using a smartphone can significantly contribute to future research and could also prove useful to the wider community such as amputees and those with pathological gait. In addition, since smartphones are available to a wide group of people, a low-cost solution for gait activity identification can be realized, without requiring the use of external sensors and circuitry.