Multivariate Time Series Prediction Research Articles

Modified local Granger causality analysis based on Peter‐Clark algorithm for multivariate time series prediction on IoT data

AbstractClimate data collected through Internet of Things (IoT) devices often contain high‐dimensional, nonlinear, and auto‐correlated characteristics, and general causality analysis methods obtain quantitative causality analysis results between variables based on conditional independence tests or Granger causality, and so forth. However, it is difficult to capture dynamic properties between variables of temporal distribution, which can obtain information that cannot be obtained by the mean detection method. Therefore, this paper proposed a new causality analysis method based on Peter‐Clark (PC) algorithm and modified local Granger causality (MLGC) analysis method, called PC‐MLGC, to reveal the causal relationships between variables and explore the dynamic properties on temporal distribution. First, the PC algorithm is applied to compute the relevant variables of each variable. Then, the results obtained in the previous stage are fed into the modified local Granger causality analysis model to explore causalities between variables. Finally, combined with the quantitative causality analysis results, the dynamic characteristic curves between variables can be obtained, and the accuracy of the causal relationship between variables can be further verified. The effectiveness of the proposed method is further demonstrated by comparing it with standard Granger causality analysis and a two‐stage causal network learning method on one benchmark dataset and two real‐world datasets.

FAFTransformer: Multivariate time series prediction method based on multi‐period feature recombination

AbstractMultivariate time series forecasting is widely used in various fields in real life. Many time series prediction models have been proposed. The current forecasting model lacks the mining of correlation between sequences based on different periods and correlation of periodical features between different periods when dealing with data. In this paper, we propose a multivariate data prediction model FAFTransformer based on the reorganization of multi‐periodic features, which first extracts the multi‐periodic information of the time series using the method of frequency domain analysis. The temporal dependencies within sequences are then captured using convolution based on different periods, and the correlations between sequences are learned by combining the multivariate attention mechanism to obtain the intra‐sequence and inter‐sequence correlations under the same period. Finally, period fusion is proposed to capture the correlation of period characteristics between different periods. The experimental results show that the model achieves the best results on multiple datasets compared to the latest seven predictive models for time‐series data.

Advanced hybrid empirical mode decomposition, convolutional neural network and long short-term memory neural network approach for predicting grain pile humidity based on meteorological inputs

Grain pile humidity prediction is beneficial to ensure food security, and establishing an effective humidity prediction model is of great significance to the field of grain storage. By taking meteorological and grain temperature data as inputs, we propose a prediction model that combines Empirical Mode Decomposition (EMD), Convolutional Neural Network (CNN), and Long Short-Term Memory Network (LSTM). The model was verified in experimental data of three different storage layers of grain piles. The prediction results show that the proposed EMD-CNN-LSTM model has better prediction accuracy than the other three comparison models: CNN-LSTM, CNN and LSTM. From the average results of the entire granary, the MAE, RMSE, and MAPE results are 0.14, 0.18, and 0.25%, respectively, and the MAE value is 44% higher than the previous research method that does not consider meteorological factors. The MAE, RMSE, and MAPE results of the CNN-LSTM method with EMD decomposition were improved by 58%, 53% and 58% respectively compared with the method without EMD decomposition. It can be concluded that taking meteorological factors as model input and integrating EMD methods can improve prediction accuracy. The constructed prediction model shows effective prediction results in different storage layers of grain pile, which provides new insights for ensuring food security and also provides valuable references for multivariate time series prediction in other fields.

Performance evaluation of multivariate deep-time convolution neural architectures for short-term electricity forecasting: Findings and failures

Deep learning (DL) models hold great promise in enhancing the decision-making abilities of electricity market participants and system operators in the short term, as they excel at capturing the intricate interdependencies and uncertainties in electricity consumption. Nevertheless, currently, the adoption of DL models for electricity prediction is not keeping up with the advancements in DL research. Within this framework, the aims of this study are to examine the probabilistic, multivariate, and multihorizon time series prediction of deep time convolution neural (DTCN) models. The effectiveness of DTCN in forecasting electricity consumption over daily, weekly and monthly time horizons is emperically assessed. We analyze both probabilistic and deterministic predictions. The evaluation data comprises real residential electricity consumption, average temperature and humidity with hourly granularity, collected over one year from a residential building. For evaluation, we considered accuracy and uncertainties, sensitivity to hyperparameters, the impact of exogeneous variables, as well as computational efficiency. Results show that the probabilistic models offer greater advantages when it comes to forecasting on multihorizons, especially for diverse datasets with non-uniform time series models. Unlike traditional models, the DTCN models demonstrated superior performance in capturing complex patterns and reducing forecast errors in short-term scenarios. However, their performance can be influenced by the sensitivity of hyperparameters. These findings can act as a benchmark for quantitatively assessing the performance of novel multivariate DL models in predicting electricity demand.

A Fuzzy C-Means Clustering-Based Hybrid Multivariate Time Series Prediction Framework With Feature Selection

A Fuzzy C-Means Clustering-Based Hybrid Multivariate Time Series Prediction Framework With Feature Selection

Prediction of Multivariate Chaotic Time Series using GRU, LSTM and RNN

Prediction of Multivariate Chaotic Time Series using GRU, LSTM and RNN

A Multivariate Time Series Prediction Method Based on Convolution-Residual Gated Recurrent Neural Network and Double-Layer Attention

A Multivariate Time Series Prediction Method Based on Convolution-Residual Gated Recurrent Neural Network and Double-Layer Attention

Spatial impact on inflation of Java Island prediction using Autoregressive Integrated Moving Average (ARIMA) and Generalized Space-Time ARIMA (GSTARIMA)

Inflation is one of macroeconomic issues in Indonesia that needs to be controlled. Inflation could happen because of widespread increases in the cost of goods and services. Annual inflation rate in Indonesia on 2008 to 2023 are quite fluctuating and several periods are not achieved inflation target yet. One of the ways to control inflation is by making predictions for the upcoming period. Java Island is the biggest contributor on economy and Gross Domestic Product (GDP) in Indonesia so it can be considered as general indicator to measure overall inflation rate of Indonesia. Thus, data used in this study is monthly inflation at each province in Java Island from January 2008 to December 2023. This study using two methods, Autoregressive Integrated Moving Average (ARIMA) for univariate time series prediction and Generalized Space-Time ARIMA (GSTARIMA) for multivariate time series prediction with a spatial factor. The results of both models will be compared to determine which model has better accuracy. Based on RMSE value, GSTARIMA model has least average RMSE value, which is 0.113 compared with ARIMA model which has average RMSE value 0.319 thus it can conclude that spatial factors addition could increase accuracy on inflation prediction in Java Island.•This paper purposes to get Java Island's inflation rate prediction to determine better policy on controlling cost of goods and services.•Best model using GSTARIMA methods is GSTARMA(1,1) with distance invese matrix that indicate that coordinate point of each location increase performance of inflation rate prediction.•The result indicate GSTARIMA has better accuracy than ARIMA for inflation prediction in Java Island based on RMSE value.

A Multivariate Time Series Prediction Method for Automotive Controller Area Network Bus Data

This study addresses the prediction of CAN bus data, a lesser-explored aspect within unsupervised anomaly detection research. We propose the Fast-Gated Attention (FGA) Transformer, a novel approach designed for accurate and efficient prediction of CAN bus data. This model utilizes a cross-attention window to optimize computational scale and feature extraction, a gated single-head attention mechanism in place of multi-head attention, and shared parameters to minimize model size. Additionally, a generalized unbiased linear attention approximation technique speeds up attention block computation. On three datasets—Car-Hacking, SynCAN, and Automotive Sensors—the FGA Transformer achieves predicted root mean square errors of 1.86 × 10−3, 3.03 × 10−3, and 30.66 × 10−3, with processing speeds of 2178, 2768, and 3062 frames per second, respectively. The FGA Transformer provides the best or comparable accuracy with a speed improvement ranging from 6 to 170 times over existing methods, underscoring its potential for CAN bus data prediction.

A novel multivariate time series forecasting dendritic neuron model for COVID-19 pandemic transmission tendency

A novel coronavirus discovered in late 2019 (COVID-19) quickly spread into a global epidemic and, thankfully, was brought under control by 2022. Because of the virus’s unknown mutations and the vaccine’s waning potency, forecasting is still essential for resurgence prevention and medical resource management. Computational efficiency and long-term accuracy are two bottlenecks for national-level forecasting. This study develops a novel multivariate time series forecasting model, the densely connected highly flexible dendritic neuron model (DFDNM) to predict daily and weekly positive COVID-19 cases. DFDNM’s high flexibility mechanism improves its capacity to deal with nonlinear challenges. The dense introduction of shortcut connections alleviates the vanishing and exploding gradient problems, encourages feature reuse, and improves feature extraction. To deal with the rapidly growing parameters, an improved variation of the adaptive moment estimation (AdamW) algorithm is employed as the learning algorithm for the DFDNM because of its strong optimization ability. The experimental results and statistical analysis conducted across three Japanese prefectures confirm the efficacy and feasibility of the DFDNM while outperforming various state-of-the-art machine learning models. To the best of our knowledge, the proposed DFDNM is the first to restructure the dendritic neuron model’s neural architecture, demonstrating promising use in multivariate time series prediction. Because of its optimal performance, the DFDNM may serve as an important reference for national and regional government decision-makers aiming to optimize pandemic prevention and medical resource management. We also verify that DFDMN is efficiently applicable not only to COVID-19 transmission prediction, but also to more general multivariate prediction tasks. It leads us to believe that it might be applied as a promising prediction model in other fields.

NPP accident prevention: Integrated neural network for coupled multivariate time series prediction based on PSO and its application under uncertainty analysis for NPP data

Due to the requirement of a cleaner, more sustainable form of energy production and the rapid development of DT, nuclear energy needs to be digital transformed. This paper aims at the time series prediction module of DT in NPP. Despite the widespread application of time series forecasting in various domains, the inherent uncertainties within the data and the selection of neural network hyperparameters pose a formidable challenge to accurate predictions. This study proposes a coupled multivariate prediction method and conducts a comparative analysis of three popular time series forecasting models: LSTM, CNN-LSTM, and Transformer. PSO hyperparameter optimization is also employed, accompanied by multiple error calculation metrics for model evaluation. It is noteworthy that the experimental data employed in this study are derived from the real operation process of a Gen II + reactor in the Chinese mainland, which is in the off-site power loss accident condition. Simultaneously, adopting a smaller training dataset proportion contributes to striking a balance among resource utilization, model performance, and experimental efficiency in the research. Experimental results unequivocally demonstrate that among these models, Transformer excels in multi-input multi-output time series forecasting. Moreover, uncertainty analysis using Bayesian generates a forecast band, proving the robustness of Transformer.

A Divide-and-Rule Combined Learning Method for Truly Multivariate Time Series Prediction

Multivariate time series prediction is a significant research area that aims to forecast future values based on past observations. Deep learning models with attention mechanisms have shown good predictive performance by emphasizing optimal-related sequences in the target series. However, these models ignore mutation information of nontarget sequences and the long short-term dependencies. To this end, a divide-and-rule combined learning method is proposed to address these limitations, which uses differentiated feature extractors to process different implicit features. First, we design a spatial and temporal information extractor to extract the time-dimensional feature information in the separation stage. Then, a multivariate mutation information extractor is constructed by convolution and maximum pooling layer to capture mutation information of nontarget sequences. Subsequently, the decoder component of the encoder-decoder model extracts long short-term dependencies while preserving the information of the target sequence to be predicted. Finally, in the cooperation stage, a feature fusion method based on a point attention mechanism is proposed, which can assign individual weights to each feature point and enhance the ability to focus on local areas. Experimental results on five real datasets in different domains show that the proposed method has better predictive performance compared to other baseline models.

A long-term multivariate time series prediction model for dissolved oxygen

Accurate and efficient long-term prediction of marine dissolved oxygen (DO) is crucial for the sustainable development of aquaculture. However, the multidimensional time dependency and lag effects of marine chemical variables present significant challenges when handling multiple inputs in univariate long-term prediction tasks. To address these issues, we designed a multivariate time-series long-term prediction model (LMFormer) based on the Transformer architecture. The proposed multivariate time-series decomposition strategy effectively leverages the feature information of prediction variables at different scales, thereby reducing the loss of critical information. Additionally, a dynamic variable selection strategy based on a gating mechanism was designed to optimize the collinearity problem in the multivariate data feature extraction process. Finally, an efficient two-stage attention architecture is proposed to effectively capture the long-range dependencies between dynamic features. This study conducted high-precision 7-day advance DO long-term predictions in two case studies, the environmentally stable Shandong Peninsula in China and the San Juan Islands in the United States, which are affected by extreme conditions such as ocean currents. The experimental results demonstrate the superior prediction performance and generalizability of the designed model. In the Shandong Peninsula case, the mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and Kling–Gupta efficiency (KGE) reached 0.0159, 0.126, 0.9743, and 0.9625, respectively. In the San Juan Islands case, the MAE was reduced by an average of 42.34% compared to that of the baseline model, the RMSE was reduced by an average of 24.57%, the R2 increased by 22.54%, and the KGE improved by an average of 12.04%. Overall, the proposed prediction model effectively achieves long-term prediction of multivariate marine chemical data, providing valuable references for sustainable management and decision-making in aquaculture.

A common feature-driven prediction model for multivariate time series data

Multivariate time series data contain a variety of common features that are difficult to extract, among which the sudden irregular fluctuation trend, the trend feature of large fluctuation amplitude, and the long-term dependency relationship in the time series have an important impact on the accuracy of the prediction model. To predict non-stationary trends in large-scale data accurately using the common characteristics of multivariate time series. A novel generalised forecasting model NeA3L based on common features of time series is developed. The NeA3L model utilizes multiple independent parallel feature extraction modules to obtain the common features of multivariate time series. It utilizes the three-layer iterative structure to deal with sudden irregular fluctuation patterns. NeA3L optimizes the network structure to realize the heterogeneous codec structure. It applies the attention mechanism between the encoder and the decoder to accomplish the multivariate prediction of the multivariate time series, which has good stability for predicting various types of multivariate data. A comparison of the NeA3L model with nine current time series prediction methods on four publicly available datasets shows that the NeA3L model outperforms the current methods in various evaluation metrics. The RMSE is improved by 2.3 %, 26.5 %, 28.9 %, and 20.84 % on average, respectively. The NeA3L model can be universally applicable to the field of multivariate time series prediction, which is important for optimizing the intelligent management and decision-making.

Functional Relation Field: A Model-Agnostic Framework for Multivariate Time Series Forecasting

In multivariate time series forecasting, the most popular strategy for modeling the relationship between multiple time series is the construction of graph, where each time series is represented as a node and related nodes are connected by edges. However, the relationship between multiple time series is typically complicated, e.g. the sum of outflows from upstream nodes may be equal to the inflows of downstream nodes. Such relations widely exist in many real-world scenarios for multivariate time series forecasting, yet are far from well studied. In these cases, graph might be insufficient for modeling the complex dependency between nodes. To this end, we explore a new framework to model the inter-node relationship in a more precise way based our proposed inductive bias, Functional Relation Field, where a group of functions parameterized by neural networks are learned to characterize the dependency between multiple time series. Essentially, these learned functions then form a “field”, i.e. a particular set of constraints, to regularize the training loss of the backbone prediction network and enforce the inference process to satisfy these constraints. Since our framework introduces the relationship bias in a data-driven manner, it is flexible and model-agnostic such that it can be applied to any existing multivariate time series prediction networks for boosting performance. The experiment is conducted on one toy dataset to show our approach can well recover the true constraint relationship between nodes. And various real-world datasets are also considered with different backbone prediction networks. Results show that the prediction error can be reduced remarkably with the aid of the proposed framework.

Enhanced Multi-variate Time Series Prediction Through Statistical-Deep Learning Integration: The VAR-Stacked LSTM Model

Enhanced Multi-variate Time Series Prediction Through Statistical-Deep Learning Integration: The VAR-Stacked LSTM Model

Integrating gated recurrent unit in graph neural network to improve infectious disease prediction: an attempt.

This study focuses on enhancing the precision of epidemic time series data prediction by integrating Gated Recurrent Unit (GRU) into a Graph Neural Network (GNN), forming the GRGNN. The accuracy of the GNN (Graph Neural Network) network with introduced GRU (Gated Recurrent Units) is validated by comparing it with seven commonly used prediction methods. The GRGNN methodology involves multivariate time series prediction using a GNN (Graph Neural Network) network improved by the integration of GRU (Gated Recurrent Units). Additionally, Graphical Fourier Transform (GFT) and Discrete Fourier Transform (DFT) are introduced. GFT captures inter-sequence correlations in the spectral domain, while DFT transforms data from the time domain to the frequency domain, revealing temporal node correlations. Following GFT and DFT, outbreak data are predicted through one-dimensional convolution and gated linear regression in the frequency domain, graph convolution in the spectral domain, and GRU (Gated Recurrent Units) in the time domain. The inverse transformation of GFT and DFT is employed, and final predictions are obtained after passing through a fully connected layer. Evaluation is conducted on three datasets: the COVID-19 datasets of 38 African countries and 42 European countries from worldometers, and the chickenpox dataset of 20 Hungarian regions from Kaggle. Metrics include Average Root Mean Square Error (ARMSE) and Average Mean Absolute Error (AMAE). For African COVID-19 dataset and Hungarian Chickenpox dataset, GRGNN consistently outperforms other methods in ARMSE and AMAE across various prediction step lengths. Optimal results are achieved even at extended prediction steps, highlighting the model's robustness. GRGNN proves effective in predicting epidemic time series data with high accuracy, demonstrating its potential in epidemic surveillance and early warning applications. However, further discussions and studies are warranted to refine its application and judgment methods, emphasizing the ongoing need for exploration and research in this domain.

DSTN: Dynamic Spatio-Temporal Network for Early Fault Warning in Chemical Processes

Multivariate time series prediction, especially in early fault warning for chemical processes, poses significant challenges. The advent of graph neural network (GNN) method has made breakthroughs in this domain by enabling the processing of topological data. However, the traditional methods suffer from the issue of over-smoothing and inability to capture intricate multi-scale spatio-temporal dependencies. Additionally, the existing graph structures fall short in describing the complex spatial relationships among multi-stage sensors, impeding their adaptability to dynamically evolving chain reaction scenarios. To alleviate these limitations, a novel Dynamic spatio-Temporal Network for early fault warning in chemical processes, named DSTN for short, is proposed in this paper. We extract the spatial and temporal features of the time series by the designed dynamic GNN and the improved Transformer network. Then, we integrate the spatio-temporal features through the residual network. DSTN has the following advantages: (1) A one-dimensional convolutional neural network is seamlessly incorporated into the Transformer architecture for bolstering its capacity to discern both global and local features within time series. (2) The continuous sliding window and mutual information methods are employed to construct a dynamic topology graph, and a K-order adjacency matrix is designed to rectify the inefficiencies in learning weights associated with convolution kernel parameters. (3) Multiple spatio-temporal modules interconnected via residual connection to adaptively fuse multi-scale features. Experimental results demonstrate that our proposed DSTN method outperforms existing methods in terms of both performance and interpretability in early fault warning of chemical processes.

A New Auto-Regressive Multi-Variable Modified Auto-Encoder for Multivariate Time-Series Prediction: A Case Study with Application to COVID-19 Pandemics.

The SARS-CoV-2 global pandemic prompted governments, institutions, and researchers to investigate its impact, developing strategies based on general indicators to make the most precise predictions possible. Approaches based on epidemiological models were used but the outcomes demonstrated forecasting with uncertainty due to insufficient or missing data. Besides the lack of data, machine-learning models including random forest, support vector regression, LSTM, Auto-encoders, and traditional time-series models such as Prophet and ARIMA were employed in the task, achieving remarkable results with limited effectiveness. Some of these methodologies have precision constraints in dealing with multi-variable inputs, which are important for problems like pandemics that require short and long-term forecasting. Given the under-supply in this scenario, we propose a novel approach for time-series prediction based on stacking auto-encoder structures using three variations of the same model for the training step and weight adjustment to evaluate its forecasting performance. We conducted comparison experiments with previously published data on COVID-19 cases, deaths, temperature, humidity, and air quality index (AQI) in São Paulo City, Brazil. Additionally, we used the percentage of COVID-19 cases from the top ten affected countries worldwide until May 4th, 2020. The results show 80.7% and 10.3% decrease in RMSE to entire and test data over the distribution of 50 trial-trained models, respectively, compared to the first experiment comparison. Also, model type#3 achieved 4th better overall ranking performance, overcoming the NBEATS, Prophet, and Glounts time-series models in the second experiment comparison. This model shows promising forecast capacity and versatility across different input dataset lengths, making it a prominent forecasting model for time-series tasks.

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction.

A fuzzy cognitive map (FCM) is a simple but effective tool for modeling and predicting time series. This article focuses on the problem of multivariate time series prediction (TSP), which is essential and challenging in data mining. Although several FCM-based approaches have been designed to solve this problem, their feature extraction module designed for single mode falls short in capturing the nonlinear spatiotemporal dependencies among variates, thereby resulting in low prediction accuracy in forecasting multivariate time series, which shows that the single mode learning is not enough. Therefore, in this article, we propose a joint spatiotemporal feature learning framework for multivariate TSP, where a mix-resolution spatial module consisting of multiple sparse autoencoders (SAEs) is designed to extract the feature series with different spatial resolutions, and a mix-order spatiotemporal module concluding multiple high-order FCMs (HFCMs) is designed to model the spatiotemporal dynamics of these feature series. Finally, the outputs of the two modules are concatenated to predict future values. We refer to this framework as the spatiotemporal FCM (STFCM). Especially, an efficient learning algorithm is designed to update the integral weights of STFCM based on the batch gradient descent algorithm when it deems necessary. We validate the performance of the STFCM on four real-world datasets. Compared with the existing state-of-the-art (SOTA) methods, the experimental results not only show the advantages of the two designed modules in the STFCM but also show the excellent performance of the STFCM.