Multivariate Time Series Data Research Articles

Ultra-Short-Term Offshore Wind Power Prediction Based on PCA-SSA-VMD and BiLSTM.

In order to realize the economic dispatch and safety stability of offshore wind farms, and to address the problems of strong randomness and strong time correlation in offshore wind power forecasting, this paper proposes a combined model of principal component analysis (PCA), sparrow algorithm (SSA), variational modal decomposition (VMD), and bidirectional long- and short-term memory neural network (BiLSTM). Firstly, the multivariate time series data were screened using the principal component analysis algorithm (PCA) to reduce the data dimensionality. Secondly, the variable modal decomposition (VMD) optimized by the SSA algorithm was applied to adaptively decompose the wind power time series data into a collection of different frequency components to eliminate the noise signals in the original data; on this basis, the hyperparameters of the BiLSTM model were optimized by integrating SSA algorithm, and the final power prediction value was obtained. Ultimately, the verification was conducted through simulation experiments; the results show that the model proposed in this paper effectively improves the prediction accuracy and verifies the effectiveness of the prediction model.

Multivariate Time-Series Representation Learning via Hierarchical Correlation Pooling Boosted Graph Neural Network

Representation learning is vital for the performance of Multivariate Time Series (MTS) related tasks. Given high-dimensional MTS data, researchers generally rely on deep learning (DL) models to learn representative features. Among them, the methods that can capture the spatial-temporal dependencies within MTS data generally achieve better performance. However, they ignored hierarchical relations and the dynamic property within MTS data, hindering their performance. To address these problems, we propose a Hierarchical Correlation Pooling boosted graph neural network (HierCorrPool) for MTS data representation learning. First, we propose a novel correlation pooling scheme to learn and capture hierarchical correlations between sensors. Meanwhile, a new assignment matrix is designed to ensure the effective learning of hierarchical correlations by adaptively combining both sensor features and correlations. Second, we learn sequential graphs to represent the dynamic property within MTS data, so that this property can be captured for learning decent representations. We conducted extensive experiments to test our model on various MTS tasks, including remaining useful life prediction, human activity recognition, and sleep stage classification. Experimental results have shown the effectiveness of our proposed model.

Coupled Attention Networks for Multivariate Time Series Anomaly Detection

Multivariate time series anomaly detection (MTAD) plays a vital role in a wide variety of real-world application domains. Over the past few years, MTAD has attracted rapidly increasing attention from both academia and industry. Many deep learning and graph learning models have been developed for effective anomaly detection in multivariate time series data, which enable advanced applications such as smart surveillance and risk management with unprecedented capabilities. Nevertheless, MTAD is facing critical challenges deriving from the dependencies among sensors and variables, which often change over time. To address this issue, we propose a coupled attention-based neural network framework (CAN) for anomaly detection in multivariate time series data featuring dynamic variable relationships. We combine adaptive graph learning methods with graph attention to generate a global-local graph that can represent both global correlations and dynamic local correlations among sensors. To capture inter-sensor relationships and temporal dependencies, a convolutional neural network based on the global-local graph is integrated with a temporal self-attention module to construct a coupled attention module. In addition, we develop a multilevel encoder-decoder architecture that accommodates reconstruction and prediction tasks to better characterize multivariate time series data. Extensive experiments on real-world datasets have been conducted to evaluate the performance of the proposed CAN approach, and the results show that CAN significantly outperforms state-of-the-art baselines.

Distributional Drift Adaptation With Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting.

Due to the nonstationary nature, the distribution of real-world multivariate time series (MTS) changes over time, which is known as distribution drift. Most existing MTS forecasting models greatly suffer from distribution drift and degrade the forecasting performance over time. Existing methods address distribution drift via adapting to the latest arrived data or self-correcting per the meta knowledge derived from future data. Despite their great success in MTS forecasting, these methods hardly capture the intrinsic distribution changes, especially from a distributional perspective. Accordingly, we propose a novel framework temporal conditional variational autoencoder (TCVAE) to model the dynamic distributional dependencies over time between historical observations and future data in MTSs and infer the dependencies as a temporal conditional distribution to leverage latent variables. Specifically, a novel temporal Hawkes attention (THA) mechanism represents temporal factors that subsequently fed into feedforward networks to estimate the prior Gaussian distribution of latent variables. The representation of temporal factors further dynamically adjusts the structures of Transformer-based encoder and decoder to distribution changes by leveraging a gated attention mechanism (GAM). Moreover, we introduce conditional continuous normalization flow (CCNF) to transform the prior Gaussian to a complex and form-free distribution to facilitate flexible inference of the temporal conditional distribution. Extensive experiments conducted on six real-world MTS datasets demonstrate the TCVAE's superior robustness and effectiveness over the state-of-the-art MTS forecasting baselines. We further illustrate the TCVAE applicability through multifaceted case studies and visualization in real-world scenarios.

Multistage Graph Convolutional Network With Spatial Attention for Multivariate Time Series Imputation.

In multivariate time series (MTS) analysis, data loss is a critical issue that degrades analytical model performance and impairs downstream tasks such as structural health monitoring (SHM) and traffic flow monitoring. In real-world applications, MTS is usually collected by multiple types of sensors, making MTS and correlations between variates heterogeneous. However, existing MTS imputation methods overlook the heterogeneous correlations by manipulating heterogeneous MTS as a homogeneous entity, leading to inaccurate imputation results. Besides, correlations between different data types vary due to ever-changing environmental conditions, forming dynamic correlations in MTS. How to properly learn the hidden correlation from heterogeneous MTS for accurate data imputation remains unresolved. To solve the problem, we propose a multistage graph convolutional network with spatial attention (MSA-GCN). In the first stage, we decompose heterogeneous MTS into several clusters with homogeneous data collected from identical sensor types and learn intracluster correlations. Then, we devise a GCN with spatial attention to explore dynamic intercluster correlations, which is the second stage of MSA-GCN. In the last stage, we decode the learned features from previous stages via stacked convolutional neural networks. We jointly train these three-stage models to predict the missing data in MTS. Leveraging this multistage architecture and spatial attention mechanism makes MSA-GCN effectively learn heterogeneous and dynamic correlations among MTS, resulting in superior imputation performance. We tested MSA-GCN with the monitoring data from a large-span bridge and Wetterstation weather dataset. The results affirm its superiority over baseline models, demonstrating its enhanced accuracy in reducing imputation errors across diverse datasets.

General value functions for fault detection in multivariate time series data.

One of the greatest challenges to the automated production of goods is equipment malfunction. Ideally, machines should be able to automatically predict and detect operational faults in order to minimize downtime and plan for timely maintenance. While traditional condition-based maintenance (CBM) involves costly sensor additions and engineering, machine learning approaches offer the potential to learn from already existing sensors. Implementations of data-driven CBM typically use supervised and semi-supervised learning to classify faults. In addition to a large collection of operation data, records of faulty operation are also necessary, which are often costly to obtain. Instead of classifying faults, we use an approach to detect abnormal behaviour within the machine's operation. This approach is analogous to semi-supervised anomaly detection in machine learning (ML), with important distinctions in experimental design and evaluation specific to the problem of industrial fault detection. We present a novel method of machine fault detection using temporal-difference learning and General Value Functions (GVFs). Using GVFs, we form a predictive model of sensor data to detect faulty behaviour. As sensor data from machines is not i.i.d. but closer to Markovian sampling, temporal-difference learning methods should be well suited for this data. We compare our GVF outlier detection (GVFOD) algorithm to a broad selection of multivariate and temporal outlier detection methods, using datasets collected from a tabletop robot emulating the movement of an industrial actuator. We find that not only does GVFOD achieve the same recall score as other multivariate OD algorithms, it attains significantly higher precision. Furthermore, GVFOD has intuitive hyperparameters which can be selected based upon expert knowledge of the application. Together, these findings allow for a more reliable detection of abnormal machine behaviour to allow ideal timing of maintenance; saving resources, time and cost.

Stacked LSTM for Short-Term Wind Power Forecasting Using Multivariate Time Series Data

Stacked LSTM for Short-Term Wind Power Forecasting Using Multivariate Time Series Data

GAN-based Temporal Association Rule Mining on Multivariate Time Series Data

GAN-based Temporal Association Rule Mining on Multivariate Time Series Data

Robotic Kinesthesia: Estimating Object Geometry and Material With Robot's Haptic Senses.

Humans excel at determining the shape and material of objects through touch. Drawing inspiration from this ability, we propose a robotic system that incorporates haptic sensing capability into its artificial recognition system to jointly learn the shape and material types of an object. To achieve this, we employ a serially connected robotic arm and develop a supervised learning task that learns and classifies target surface geometry and material types using multivariate time-series data from joint torque sensors. Additionally, we propose a joint torque-to-position generation task to derive a one-dimensional surface profile based on torque measurements. Experimental results successfully validate the proposed torque-based classification and regression tasks, suggesting that a robotic system can employ haptic sensing (i.e., perceived force) from each joint to recognize material types and geometry, akin to human abilities.

Traffic flow prediction with a multi-dimensional feature input: A new method based on attention mechanisms

<abstract> <p>Accurately predicting traffic flow is an essential component of intelligent transportation systems. The advancements in traffic data collection technology have broadened the range of features that affect and represent traffic flow variations. However, solely inputting gathered features into the model without analysis might overlook valuable information, hindering the improvement of predictive performance. Furthermore, intricate dynamic relationships among various feature inputs could constrain the model's potential for further enhancement in predictive accuracy. Consequently, extracting pertinent features from datasets and modeling their mutual influence is critical in attaining heightened precision in traffic flow predictions. First, we perform effective feature extraction by considering the temporal dimension and inherent operating rules of traffic flow, culminating in Multivariate Time Series (MTS) data used as input for the model. Then, an attention mechanism is proposed based on the MTS input data. This mechanism assists the model in selecting pertinent time series for multivariate forecasting, mitigating inter-feature influence, and achieving accurate predictions through the concentration on crucial information. Finally, empirical findings from real highway datasets illustrate the enhancement of predictive accuracy attributed to the proposed features within the model. In contrast to conventional machine learning or attention-based deep learning models, the proposed attention mechanism in this study demonstrates superior accuracy and stability in MTS-based traffic flow prediction tasks.</p> </abstract>

Hemispheric Prediction of Solar Cycles 25 and 26 From Multivariate Sunspot Time-series Data via Informer Models

Hemispheric Prediction of Solar Cycles 25 and 26 From Multivariate Sunspot Time-series Data via Informer Models

MetricSifter: Feature Reduction of Multivariate Time Series Data for Efficient Fault Localization in Cloud Applications

MetricSifter: Feature Reduction of Multivariate Time Series Data for Efficient Fault Localization in Cloud Applications

Bayesian Information Criterion Analysis for Accuracy Improvement of Multivariate Time Series Data Analysis

Bayesian Information Criterion Analysis for Accuracy Improvement of Multivariate Time Series Data Analysis

Online Learning of Temporal Association Rule on Dynamic Multivariate Time Series Data

Online Learning of Temporal Association Rule on Dynamic Multivariate Time Series Data

A Machine Learning Model Based on GRU and LSTM to Predict the Environmental Parameters in a Layer House, Taking CO2 Concentration as an Example.

In a layer house, the CO2 (carbon dioxide) concentration above the upper limit can cause the oxygen concentration to be below the lower limit suitable for poultry. This leads to chronic CO2 poisoning in layers, which manifests as listlessness, reduced appetite, weak constitution, decreased production performance, and weakened resistance to disease. Regulating ventilation may ensure a suitable CO2 concentration in layer houses. Predicting the changes in CO2 concentration and regulating the CO2 concentration in advance are key to ensuring healthy large-scale breeding of layers. In recent years, machine learning and deep learning methods have been increasingly applied to this field. A CO2 prediction model for layer house is proposed based on a GRU (gated recurrent unit) and LSTM (long short-term memory). The temperature, humidity, and CO2 were determined as inputs to the model by the correlation coefficient. The datasets of the experimental layer house were continuously measured during June-July 2023, using a self-developed environmental monitor, and the monitored data were used as samples for model inputs. There were 22,000 time series data in the datasets. In this study, multivariate time series data were standardized via data pre-processing to improve model training. GRU and LSTM models were constructed. The models were trained using a training set. Then, these trained models were used to provide predictions on a test set. The prediction errors were calculated using the true values of the test set and the predicted values provided by the models. To test the performance of the model and accuracy of the predictions, predictions were made for different numbers of datasets. The results demonstrated that the combined prediction model had good generalization, stability, and convergence with high prediction accuracy. Due to the structure of the model, the stability of the LSTM model was higher than that of the GRU model, and its prediction accuracy and speed were lower than those of the GRU model. When the datasets of the GRU model were 15,000 to 17,000, The MAE of the GRU was 70.8077 to 126.7029 ppm, and the prediction time of the GRU is 16 to 24 ms. When the LSTM model's datasets were 15,000-20,000, the MAE of LSTM was 78.8596 to 136.0896 ppm, and the prediction time of the GRU was 17 to 26 ms.

AI-Assisted Fatigue and Stamina Control for Performance Sports on IMU-Generated Multivariate Times Series Datasets.

Optimal sports performance requires a balance between intensive training and adequate rest. IMUs provide objective, quantifiable data to analyze performance dynamics, despite the challenges in quantifying athlete training loads. The ability of AI to analyze complex datasets brings innovation to the monitoring and optimization of athlete training cycles. Traditional techniques rely on subjective assessments to prevent overtraining, which can lead to injury and underperformance. IMUs provide objective, quantitative data on athletes' physical status during action. AI and machine learning can turn these data into useful insights, enabling data-driven athlete performance management. With IMU-generated multivariate time series data, this paper uses AI to construct a robust model for predicting fatigue and stamina. IMUs linked to 19 athletes recorded triaxial acceleration, angular velocity, and magnetic orientation throughout repeated sessions. Standardized training included steady-pace runs and fatigue-inducing techniques. The raw time series data were used to train a supervised ML model based on frequency and time-domain characteristics. The performances of Random Forest, Gradient Boosting Machines, and LSTM networks were compared. A feedback loop adjusted the model in real time based on prediction error and bias estimation. The AI model demonstrated high predictive accuracy for fatigue, showing significant correlations between predicted fatigue levels and observed declines in performance. Stamina predictions enabled individualized training adjustments that were in sync with athletes' physiological thresholds. Bias correction mechanisms proved effective in minimizing systematic prediction errors. Moreover, real-time adaptations of the model led to enhanced training periodization strategies, reducing the risk of overtraining and improving overall athletic performance. In sports performance analytics, the AI-assisted model using IMU multivariate time series data is effective. Training can be tailored and constantly altered because the model accurately predicts fatigue and stamina. AI models can effectively forecast the beginning of weariness before any physical symptoms appear. This allows for timely interventions to prevent overtraining and potential accidents. The model shows an exceptional ability to customize training programs according to the physiological reactions of each athlete and enhance the overall training effectiveness. In addition, the study demonstrated the model's efficacy in real-time monitoring performance, improving the decision-making abilities of both coaches and athletes. The approach enables ongoing and thorough data analysis, supporting strategic planning for training and competition, resulting in optimized performance outcomes. These findings highlight the revolutionary capability of AI in sports science, offering a future where data-driven methods greatly enhance athlete training and performance management.

Explainable Artificial Intelligence Warning Model Using an Ensemble Approach for In-Hospital Cardiac Arrest Prediction: Retrospective Cohort Study.

Cardiac arrest (CA) is the leading cause of death in critically ill patients. Clinical research has shown that early identification of CA reduces mortality. Algorithms capable of predicting CA with high sensitivity have been developed using multivariate time series data. However, these algorithms suffer from a high rate of false alarms, and their results are not clinically interpretable. We propose an ensemble approach using multiresolution statistical features and cosine similarity-based features for the timely prediction of CA. Furthermore, this approach provides clinically interpretable results that can be adopted by clinicians. Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV database and the eICU Collaborative Research Database. Based on the multivariate vital signs of a 24-hour time window for adults diagnosed with heart failure, we extracted multiresolution statistical and cosine similarity-based features. These features were used to construct and develop gradient boosting decision trees. Therefore, we adopted cost-sensitive learning as a solution. Then, 10-fold cross-validation was performed to check the consistency of the model performance, and the Shapley additive explanation algorithm was used to capture the overall interpretability of the proposed model. Next, external validation using the eICU Collaborative Research Database was performed to check the generalization ability. The proposed method yielded an overall area under the receiver operating characteristic curve (AUROC) of 0.86 and area under the precision-recall curve (AUPRC) of 0.58. In terms of the timely prediction of CA, the proposed model achieved an AUROC above 0.80 for predicting CA events up to 6 hours in advance. The proposed method simultaneously improved precision and sensitivity to increase the AUPRC, which reduced the number of false alarms while maintaining high sensitivity. This result indicates that the predictive performance of the proposed model is superior to the performances of the models reported in previous studies. Next, we demonstrated the effect of feature importance on the clinical interpretability of the proposed method and inferred the effect between the non-CA and CA groups. Finally, external validation was performed using the eICU Collaborative Research Database, and an AUROC of 0.74 and AUPRC of 0.44 were obtained in a general intensive care unit population. The proposed framework can provide clinicians with more accurate CA prediction results and reduce false alarm rates through internal and external validation. In addition, clinically interpretable prediction results can facilitate clinician understanding. Furthermore, the similarity of vital sign changes can provide insights into temporal pattern changes in CA prediction in patients with heart failure-related diagnoses. Therefore, our system is sufficiently feasible for routine clinical use. In addition, regarding the proposed CA prediction system, a clinically mature application has been developed and verified in the future digital health field.

Memetic segmentation based on variable lag aware for multivariate time series

The memetic segmentation algorithm has been proposed to obtain precise segmentation results through efficient local refinement. However, this algorithm overlooks the consideration of time lags among variables or spatial locations in multivariate time series data. Although some researchers have adopted dynamic programming to eliminate time lags and obtain segmentation results iteratively, they have failed to achieve simultaneous optimization for both lag awareness and segmentation. To address this problem, we propose a memetic segmentation algorithm based on variable lag-aware for multivariate time series (MSVLAMTS), which enhances the encoding operation of the memetic algorithm to optimize both time-lag awareness and segmentation positions. Furthermore, MSVLAMTS incorporates the self-distribution of multivariate time series and the correlation between variables to construct a maximum likelihood function for fitness evaluation, and further improves the optimization strategy of the memetic algorithm. Experiment results on artificial and real datasets demonstrate that MSVLAMTS exhibits superior performance in both time-lag awareness and segmentation. It effectively resolves the challenge of variable time-lag awareness in multivariate time series data and enhances the accuracy of collaborative segmentation.

Dynamic Feature Extraction-Based Quadratic Discriminant Analysis for Industrial Process Fault Classification and Diagnosis.

This paper introduces a novel method for enhancing fault classification and diagnosis in dynamic nonlinear processes. The method focuses on dynamic feature extraction within multivariate time series data and utilizes dynamic reconstruction errors to augment the feature set. A fault classification procedure is then developed, using the weighted maximum scatter difference (WMSD) dimensionality reduction criterion and quadratic discriminant analysis (QDA) classifier. This method addresses the challenge of high-dimensional, sample-limited fault classification, offering early diagnosis capabilities for online samples with smaller amplitudes than the training set. Validation is conducted using a cold rolling mill simulation model, with performance compared to classical methods like linear discriminant analysis (LDA) and kernel Fisher discriminant analysis (KFD). The results demonstrate the superiority of the proposed method for reliable industrial process monitoring and fault diagnosis.

Statistical inference for dependence networks in topological data analysis.

Topological data analysis (TDA) provide tools that are becoming increasingly popular for analyzing multivariate time series data. One key aspect in analyzing multivariate time series is dependence between components. One application is on brain signal analysis. In particular, various dependence patterns in brain networks may be linked to specific tasks and cognitive processes. These dependence patterns may be altered by various neurological and cognitive impairments such as Alzheimer's and Parkinson's diseases, as well as attention deficit hyperactivity disorder (ADHD). Because there is no ground-truth with known dependence patterns in real brain signals, testing new TDA methods on multivariate time series is still a challenge. Our goal here is to develop novel statistical inference procedures via simulations. Simulations are useful for generating some null distributions of a test statistic (for hypothesis testing), forming confidence regions, and for evaluating the performance of proposed TDA methods. To the best of our knowledge, there are no methods that simulate multivariate time series data with potentially complex user-specified connectivity patterns. In this paper we present a novel approach to simulate multivariate time series with specific number of cycles/holes in its dependence network. Furthermore, we also provide a procedure for generating higher dimensional topological features.