Inertial Signals Research Articles

Wavelet Encoding Network for Inertial Signal Enhancement via Feature Supervision

Wavelet Encoding Network for Inertial Signal Enhancement via Feature Supervision

Feature-Level Fusion Multi-Sensor Aggregation Temporal Network for Smartphone-Based Human Activity Recognition

Smartphone-based Human Activity Recognition (HAR) identifies human movements using inertial signals gathered from multiple smartphone sensors. Generally, these signals are stacked as one (data-level fusion) and fed into deep learning algorithms for feature extractions. This research studies feature-level fusion, individually processing inertial signals from each sensor, and proposes a lightweight deep temporal learning model, Feature-Level Fusion Multi-Sensor Aggregation Temporal Network (FLF-MSATN), that performs feature extraction on inertial signals from each sensor separately. The raw signals, segmented into equally sized time windows, are passed into individual Dilated-Pooled Convolutional Heads (DPC Heads) for temporal feature analysis. Each DPC Head has a spatiotemporal block containing dilated causal convolutions and average pooling, to extract underlying patterns. The DPC Heads’ outputs are concatenated and passed into a Global Average Pooling layer to generate a condensed confidence map before activity classification. FLF-MSATN is assessed using a subject-independent protocol on a publicly available HAR dataset, UCI HAR, and a self-collected HAR dataset, achieving 96.67% and 82.70% accuracies, respectively. A Data-Level Fusion MSATN is built to compare and verify the model performance attained by the proposed FLF-MSATN. The empirical results show that implementing FLF-MSATN enhances the accuracy by ~3.4% for UCI HAR and ~9.68% for self-collected datasets.

A method for the synchronization of inertial sensor signals and local field potentials from deep brain stimulation systems

Objective. Recent innovative neurostimulators allow recording local field potentials (LFPs) while performing motor tasks monitored by wearable sensors. Inertial sensors can provide quantitative measures of motor impairment in people with subthalamic nucleus deep brain stimulation. To the best of our knowledge, there is no validated method to synchronize inertial sensors and neurostimulators without an additional device. This study aims to define a new synchronization method to analyze disease-related brain activity patterns during specific motor tasks and evaluate how LFPs are affected by stimulation and medication. Approach. Fourteen male subjects treated with subthalamic nucleus deep brain stimulation were recruited to perform motor tasks in four different medication and stimulation conditions. In each condition, a synchronization protocol was performed consisting of taps on the implanted neurostimulator, which produces artifacts in the LFPs that a nearby inertial sensor can simultaneously record. Main results. In 64% of the recruited subjects, induced artifacts were detected at least in one condition. Among those subjects, 83% of the recordings could be synchronized offline analyzing LFPs and wearables data. The remaining recordings were synchronized by video analysis. Significance. The proposed synchronization method does not require an external system (e.g., TENS electrodes) and can be easily integrated into clinical practice. The procedure is simple and can be carried out in a short time. A proper and simple synchronization will also be useful to analyze subthalamic neural activity in the presence of specific events (e.g., freezing of gait events) to identify predictive biomarkers.

Digital-analog hybrid control of an inverted pendulum robot based on memristor

Abstract Single inertial measurement unit sensor signals are inaccurate, and traditional digital processing systems with separate storage and computation architectures cannot directly handle analog sensor signals, leading to significant delays and high power consumption. In that case, the control capability for self-balancing robots has been limited. In this work, a three-terminal MoS2-based memristor device was fabricated and successfully integrated into a memristor-based hybrid control inverted pendulum robot system. This system achieved continuous multi-sensor analog data fusion computations and analog PD control computations by combining the least squares method and a gradient descent algorithm based on a reward mechanism in digital circuits to control the memristors dynamically. Additionally, simulation testing and real, inverted pendulum robot control confirmed the efficacy of its data fusion and the accuracy of its PD control. The comparative experiments demonstrated that the hybrid control system based on memristors was approximately 86.41 mJ lower than the digital system. This system holds significant importance for the processing of large-scale sensor data and robot control.

HARWE: A multi-modal large-scale dataset for context-aware human activity recognition in smart working environments

In recent years, deep neural networks (DNNs) have provided high performances for various tasks, such as human activity recognition (HAR), in view of their end-to-end training process between the input data and output labels. However, the performances of the DNNs are highly dependent on the availability of large-scale data for their training processes. In this paper, we propose a novel dataset for the task of HAR, in which the labels are specified for the working environments (WE). Our proposed dataset, namely HARWE, considers multiple signal modalities, including visual signal, audio signal, inertial sensor signals, and biological signals, that are acquired using four different electronic devices. Furthermore, our HARWE dataset is acquired from a large number of participants while considering the realistic disturbances that can occur in the wild. Our HARWE data is context-driven, which means there exist a number of labels in it that even though they are correlated with each other, they have contextual differences. A deep conventional multi-modal neural network provides an accuracy of 99.06% and 68.60%, for the cases of the easy and difficult settings of our dataset, respectively, which indicates its applicability for the task of human activity recognition.

Human Activity Recognition Algorithm with Physiological and Inertial Signals Fusion: Photoplethysmography, Electrodermal Activity, and Accelerometry.

Inertial signals are the most widely used signals in human activity recognition (HAR) applications, and extensive research has been performed on developing HAR classifiers using accelerometer and gyroscope data. This study aimed to investigate the potential enhancement of HAR models through the fusion of biological signals with inertial signals. The classification of eight common low-, medium-, and high-intensity activities was assessed using machine learning (ML) algorithms, trained on accelerometer (ACC), blood volume pulse (BVP), and electrodermal activity (EDA) data obtained from a wrist-worn sensor. Two types of ML algorithms were employed: a random forest (RF) trained on features; and a pre-trained deep learning (DL) network (ResNet-18) trained on spectrogram images. Evaluation was conducted on both individual activities and more generalized activity groups, based on similar intensity. Results indicated that RF classifiers outperformed corresponding DL classifiers at both individual and grouped levels. However, the fusion of EDA and BVP signals with ACC data improved DL classifier performance compared to a baseline DL model with ACC-only data. The best performance was achieved by a classifier trained on a combination of ACC, EDA, and BVP images, yielding F1-scores of 69 and 87 for individual and grouped activity classifications, respectively. For DL models trained with additional biological signals, almost all individual activity classifications showed improvement (p-value < 0.05). In grouped activity classifications, DL model performance was enhanced for low- and medium-intensity activities. Exploring the classification of two specific activities, ascending/descending stairs and cycling, revealed significantly improved results using a DL model trained on combined ACC, BVP, and EDA spectrogram images (p-value < 0.05).

Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations.

Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.

Quantitative Analysis of Mother Wavelet Function Selection for Wearable Sensors-Based Human Activity Recognition.

Recent advancements in the Internet of Things (IoT) wearable devices such as wearable inertial sensors have increased the demand for precise human activity recognition (HAR) with minimal computational resources. The wavelet transform, which offers excellent time-frequency localization characteristics, is well suited for HAR recognition systems. Selecting a mother wavelet function in wavelet analysis is critical, as optimal selection improves the recognition performance. The activity time signals data have different periodic patterns that can discriminate activities from each other. Therefore, selecting a mother wavelet function that closely resembles the shape of the recognized activity's sensor (inertial) signals significantly impacts recognition performance. This study uses an optimal mother wavelet selection method that combines wavelet packet transform with the energy-to-Shannon-entropy ratio and two classification algorithms: decision tree (DT) and support vector machines (SVM). We examined six different mother wavelet families with different numbers of vanishing points. Our experiments were performed on eight publicly available ADL datasets: MHEALTH, WISDM Activity Prediction, HARTH, HARsense, DaLiAc, PAMAP2, REALDISP, and HAR70+. The analysis demonstrated in this paper can be used as a guideline for optimal mother wavelet selection for human activity recognition.

Adaptive Virtual Inertial Control and Virtual Droop Control Coordinated Control Strategy for Hybrid Energy Storage Taking into Account State of Charge Optimization

For energy-storage-assisting conventional units to participate in the primary frequency regulation of a power system, firstly, based on the frequency regulation mechanism of virtual inertial control (VIC) and virtual droop control (VDC) of energy storage, we analyze the effect of the action timing of energy storage on the frequency deviation of the grid under two control methods and put forward a reasonable combination of the two control methods; on this basis, we also put forward hybrid energy storage adaptive VIC and VDC based on the demand of VIC and VDC on the power and capacity of energy storage. On this basis, based on the demand of VIC and VDC on the power and capacity of energy storage, a hybrid energy storage adaptive VIC and VDC coordinated control strategy based on supercapacitor–lithium batteries is proposed, whereby a high-power storage supercapacitor responds to inertial control signals to rapidly suppress a drop in frequency, and the high-capacity lithium battery responds to droop control signals to perform long-time droop control. The high-capacity lithium battery responds to the sagging control signal and is used to perform a long-time sagging power response; finally, in order to avoid the state of charge (SOC) of energy storage falling into a low/high working condition and losing the subsequent frequency regulation ability, an adaptive power control strategy of energy storage based on the improved logistic function is proposed. The simulation results show that under typical load disturbance, the SOC level of the proposed strategy increases by 19.17% and 30.16%, respectively, compared with that of the single-lithium strategy and no energy storage, and the SOC level of the supercapacitor and lithium battery increases by 29.4% and 2.1%, respectively, compared with that of logistic optimization.

Wavelet Dynamic Selection Network for Inertial Sensor Signal Enhancement

As attitude and motion sensing components, inertial sensors are widely used in various portable devices, covering consumer electronics, sports health, aerospace, etc. But the severe intrinsic errors of inertial sensors heavily restrain their function implementation, especially the advanced functionality, including motion trajectory recovery and motion semantic recognition, which attracts considerable attention. As a mainstream signal processing method, wavelet is hailed as the mathematical microscope of signal due to the plentiful and diverse wavelet basis functions. However, complicated noise types and application scenarios of inertial sensors make selecting wavelet basis perplexing. To this end, we propose a wavelet dynamic selection network (WDSNet), which intelligently selects the appropriate wavelet basis for variable inertial signals. In addition, existing deep learning architectures excel at extracting features from input data but neglect to learn the characteristics of target categories, which is essential to enhance the category awareness capability, thereby improving the selection of wavelet basis. Therefore, we propose a category representation mechanism (CRM), which enables the network to extract and represent category features without increasing trainable parameters. Furthermore, CRM transforms the common fully connected network into category representations, which provide closer supervision to the feature extractor than the far and trivial one-hot classification labels. We call this process of imposing interpretability on a network and using it to supervise the feature extractor the feature supervision mechanism, and its effectiveness is demonstrated experimentally and theoretically in this paper. The enhanced inertial signal can perform impracticable tasks with regard to the original signal, such as trajectory reconstruction. Both quantitative and visual results show that WDSNet outperforms the existing methods. Remarkably, WDSNet, as a weakly-supervised method, achieves the state-of-the-art performance of all the compared fully-supervised methods.

Capability of Machine Learning Algorithms to Classify Safe and Unsafe Postures during Weight Lifting Tasks Using Inertial Sensors.

Occupational ergonomics aims to optimize the work environment and to enhance both productivity and worker well-being. Work-related exposure assessment, such as lifting loads, is a crucial aspect of this discipline, as it involves the evaluation of physical stressors and their impact on workers' health and safety, in order to prevent the development of musculoskeletal pathologies. In this study, we explore the feasibility of machine learning (ML) algorithms, fed with time- and frequency-domain features extracted from inertial signals (linear acceleration and angular velocity), to automatically and accurately discriminate safe and unsafe postures during weight lifting tasks. The signals were acquired by means of one inertial measurement unit (IMU) placed on the sternums of 15 subjects, and subsequently segmented to extract several time- and frequency-domain features. A supervised dataset, including the extracted features, was used to feed several ML models and to assess their prediction power. Interesting results in terms of evaluation metrics for a binary safe/unsafe posture classification were obtained with the logistic regression algorithm, which outperformed the others, with accuracy and area under the receiver operating characteristic curve values of up to 96% and 99%, respectively. This result indicates the feasibility of the proposed methodology-based on a single inertial sensor and artificial intelligence-to discriminate safe/unsafe postures associated with load lifting activities. Future investigation in a wider study population and using additional lifting scenarios could confirm the potentiality of the proposed methodology, supporting its applicability in the occupational ergonomics field.

Attention-Based CNN Fusion Model for Emotion Recognition During Walking Using Discrete Wavelet Transform on EEG and Inertial Signals

Attention-Based CNN Fusion Model for Emotion Recognition During Walking Using Discrete Wavelet Transform on EEG and Inertial Signals

Detection and Validation of Macro-Activities in Human Inertial Signals Using Graph Link Prediction.

With the continuous development of new wearable devices, sensor-based human activity recognition is enjoying enormous popularity in research and industry. The signals from inertial sensors allow for the detection, classification, and analysis of human activities such as jogging, cycling, or swimming. However, human activity recognition is often limited to basic activities that occur in short, predetermined periods of time (sliding windows). Complex macro-activities, such as multi-step sports exercises or multi-step cooking recipes, are still only considered to a limited extent, while some works have investigated the classification of macro-activities, the automated understanding of how the underlying micro-activities interact remains an open challenge. This study addresses this gap through the application of graph link prediction, a well-known concept in graph theory and graph neural networks (GNNs). To this end, the presented approach transforms micro-activity sequences into micro-activity graphs that are then processed with a GNN. The evaluation on two derived real-world data sets shows that graph link prediction enables the accurate identification of interactions between micro-activities and the precise validation of composite macro-activities based on learned graph embeddings. Furthermore, this work shows that GNNs can benefit from positional encodings in sequence recognition tasks.

A model for new media data mining and analysis in online English teaching using long short-term memory (LSTM) network.

To maintain a harmonious teacher-student relationship and enable educators to gain a more insightful understanding of students' learning progress, this study collects data from learners utilizing the software through a network platform. These data are mainly formed by the user's learning characteristics, combined with the screen lighting time, built-in inertial sensor attitude, signal strength, network strength and other multi-dimensional characteristics to form the learning observation value, so as to analyze the corresponding learning state, so that teachers can carry out targeted teaching improvement. The article introduces an intelligent classification approach for learning time series, leveraging long short-term memory (LSTM) as the foundation of a deep network model. This model intelligently recognizes the learning status of students. The test results demonstrate that the proposed model achieves highly precise time series recognition using relatively straightforward features. This precision, exceeding 95%, is of significant importance for future applications in learning state recognition, aiding teachers in gaining an intelligent grasp of students' learning status.

Hand Movement Recognition and Salient Tremor Feature Extraction With Wearable Devices in Parkinson’s Patients

Tremor is one of the earliest signs of Parkinson’s disease (PD), which seriously disrupts patients’ daily lives. It is important to study upper limb tremors quantitatively to control PD progression. In this study, surface electromyography (sEMG) signals from wearable devices are used to recognize rest, posture, and kinetic tremor action from 6 upper-limb clinical actions and to quantify features of tremors. A multivariable time series classification model (MTSCM) based on fully convolutional networks and a long short-term memory network is proposed to recognize tremor actions. MTSCM achieves a high degree of accuracy both on the left-hand and right-hand data sets for tremor actions. An improved Hilbert-Huang transform (HHT) method is proposed to decompose the inertial signals of tremor actions to obtain tremor components. Using the improved HHT, tremor and motion components can be decomposed effectively. In addition, 53 features are extracted from inertial and sEMG signals, and a canonical correlation analysis is used to determine the correlation between features and MDS-UPDRS scores. Several of the relevant characteristics are related to MDS-UPDRS scores, notably the dominant frequency and amplitude of the tremor component are significantly correlated (p < 0.01) with tremor scores. Detecting upper-limb clinical tremors in PD patients using wearable sensors is feasible according to our findings.

Wrist-to-Tibia/Shoe Inertial Measurement Results Translation Using Neural Networks.

Most of the established gait evaluation methods use inertial sensors mounted in the lower limb area (tibias, ankles, shoes). Such sensor placement gives good results in laboratory conditions but is hard to apply in everyday scenarios due to the sensors' fragility and the user's comfort. The paper presents an algorithm that enables translation of the inertial signal measurements (acceleration and angular velocity) registered with a wrist-worn sensor to signals, which would be obtained if the sensor was worn on a tibia or a shoe. Four different neural network architectures are considered for that purpose: Dense and CNN autoencoders, a CNN-LSTM hybrid, and a U-Net-based model. The performed experiments have shown that the CNN autoencoder and U-Net can be successfully applied for inertial signal translation purposes. Estimating gait parameters based on the translated signals yielded similar results to those obtained based on shoe-sensor signals.

A hybrid TCN-GRU model for classifying human activities using smartphone inertial signals.

Recognising human activities using smart devices has led to countless inventions in various domains like healthcare, security, sports, etc. Sensor-based human activity recognition (HAR), especially smartphone-based HAR, has become popular among the research community due to lightweight computation and user privacy protection. Deep learning models are the most preferred solutions in developing smartphone-based HAR as they can automatically capture salient and distinctive features from input signals and classify them into respective activity classes. However, in most cases, the architecture of these models needs to be deep and complex for better classification performance. Furthermore, training these models requires extensive computational resources. Hence, this research proposes a hybrid lightweight model that integrates an enhanced Temporal Convolutional Network (TCN) with Gated Recurrent Unit (GRU) layers for salient spatiotemporal feature extraction without tedious manual feature extraction. Essentially, dilations are incorporated into each convolutional kernel in the TCN-GRU model to extend the kernel's field of view without imposing additional model parameters. Moreover, fewer short filters are applied for each convolutional layer to alleviate excess parameters. Despite reducing computational cost, the proposed model utilises dilations, residual connections, and GRU layers for longer-term time dependency modelling by retaining longer implicit features of the input inertial sequences throughout training to provide sufficient information for future prediction. The performance of the TCN-GRU model is verified on two benchmark smartphone-based HAR databases, i.e., UCI HAR and UniMiB SHAR. The model attains promising accuracy in recognising human activities with 97.25% on UCI HAR and 93.51% on UniMiB SHAR. Since the current study exclusively works on the inertial signals captured by smartphones, future studies will explore the generalisation of the proposed TCN-GRU across diverse datasets, including various sensor types, to ensure its adaptability across different applications.

Self-Contained Pedestrian Navigation Fusing ML-Selected GNSS Carrier Phase and Inertial Signals in Challenging Environments

Self-Contained Pedestrian Navigation Fusing ML-Selected GNSS Carrier Phase and Inertial Signals in Challenging Environments

Spatial Sensory References for Vestibular Self-Motion Perception.

While navigating through the surroundings, we constantly rely on inertial vestibular signals for self-motion along with visual and acoustic spatial references from the environment. However, the interaction between inertial cues and environmental spatial references is not yet fully understood. Here we investigated whether vestibular self-motion sensitivity is influenced by sensory spatial references. Healthy participants were administered a Vestibular Self-Motion Detection Task in which they were asked to detect vestibular self-motion sensations induced by low-intensity Galvanic Vestibular Stimulation. Participants performed this detection task with or without an external visual or acoustic spatial reference placed directly in front of them. We computed the d prime ( d ' ) as a measure of participants' vestibular sensitivity and the criterion as an index of their response bias. Results showed that the visual spatial reference increased sensitivity to detect vestibular self-motion. Conversely, the acoustic spatial reference did not influence self-motion sensitivity. Both visual and auditory spatial references did not cause changes in response bias. Environmental visual spatial references provide relevant information to enhance our ability to perceive inertial self-motion cues, suggesting a specific interaction between visual and vestibular systems in self-motion perception.

Smartphone-based evaluation of static balance and mobility in long-lasting COVID-19 patients.

SARS-CoV-2 infection can lead to a variety of persistent sequelae, collectively known as long COVID-19. Deficits in postural balance have been reported in patients several months after COVID-19 infection. The purpose of this study was to evaluate the static balance and balance of individuals with long COVID-19 using inertial sensors in smartphones. A total of 73 participants were included in this study, of which 41 had long COVID-19 and 32 served as controls. All participants in the long COVID-19 group reported physical complaints for at least 7 months after SARS-CoV-2 infection. Participants were evaluated using a built-in inertial sensor of a smartphone attached to the low back, which recorded inertial signals during a static balance and mobility task (timed up and go test). The parameters of static balance and mobility obtained from both groups were compared. The groups were matched for age and BMI. Of the 41 participants in the long COVID-19 group, 22 reported balance impairment and 33 had impaired balance in the Sharpened Romberg test. Static balance assessment revealed that the long COVID-19 group had greater postural instability with both eyes open and closed than the control group. In the TUG test, the long COVID-19 group showed greater acceleration during the sit-to-stand transition compared to the control group. The smartphone was feasible to identify losses in the balance motor control and mobility of patients with long-lasting symptomatic COVID-19 even after several months or years. Attention to the balance impairment experienced by these patients could help prevent falls and improve their quality of life, and the use of the smartphone can expand this monitoring for a broader population.