Multivariate Time Series Forecasting Research Articles

Design Trend Fuzzy Granulation-Based Three-Layer Fuzzy Cognitive Map for Long-Term Forecasting of Multivariate Time Series

Design Trend Fuzzy Granulation-Based Three-Layer Fuzzy Cognitive Map for Long-Term Forecasting of Multivariate Time Series

Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis

Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis

Rede neural de gráfico temporal espectral para previsão de preços agrícolas multivariados

ABSTRACT: Multivariate time series forecasting has an important role in many real-world domains. Especially, price prediction has always been on the focus of researchers. Yet, it is a challenging task that requires the capturing of intra-series and inter-series correlations. Most of the models in literature focus only on the correlation in temporal domain. In this paper, we have curated a new dataset from the official website of Turkish Ministry of Commerce. The dataset consists of daily prices and trade volume of vegetables and covers 1791 days between January 1, 2018 and November 26, 2022. A Spectral Temporal Graph Neural Network (StemGNN) is employed on the curated dataset and the results are given in comparison to Convolutional neural networks (CNN), Long short-term memory (LSTM) and Random Forest models. GNN architecture achieved a state-of-the-art result such as mean absolute error (MAE): 1,37 and root mean squared error (RMSE): 1.94). To our knowledge, this is one of the few studies that investigates GNN for time series analysis and the first study in architecture field.

DecMamba: Mamba Utilizing Series Decomposition for Multivariate Time Series Forecasting

DecMamba: Mamba Utilizing Series Decomposition for Multivariate Time Series Forecasting

Temporal Fusion Transformers for Enhanced Multivariate Time Series Forecasting of Indonesian Stock Prices

Temporal Fusion Transformers for Enhanced Multivariate Time Series Forecasting of Indonesian Stock Prices

Dynamic Hypergraph Structure Learning for Multivariate Time Series Forecasting

Dynamic Hypergraph Structure Learning for Multivariate Time Series Forecasting

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Distributional Drift Adaptation With Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting.

Due to the nonstationary nature, the distribution of real-world multivariate time series (MTS) changes over time, which is known as distribution drift. Most existing MTS forecasting models greatly suffer from distribution drift and degrade the forecasting performance over time. Existing methods address distribution drift via adapting to the latest arrived data or self-correcting per the meta knowledge derived from future data. Despite their great success in MTS forecasting, these methods hardly capture the intrinsic distribution changes, especially from a distributional perspective. Accordingly, we propose a novel framework temporal conditional variational autoencoder (TCVAE) to model the dynamic distributional dependencies over time between historical observations and future data in MTSs and infer the dependencies as a temporal conditional distribution to leverage latent variables. Specifically, a novel temporal Hawkes attention (THA) mechanism represents temporal factors that subsequently fed into feedforward networks to estimate the prior Gaussian distribution of latent variables. The representation of temporal factors further dynamically adjusts the structures of Transformer-based encoder and decoder to distribution changes by leveraging a gated attention mechanism (GAM). Moreover, we introduce conditional continuous normalization flow (CCNF) to transform the prior Gaussian to a complex and form-free distribution to facilitate flexible inference of the temporal conditional distribution. Extensive experiments conducted on six real-world MTS datasets demonstrate the TCVAE's superior robustness and effectiveness over the state-of-the-art MTS forecasting baselines. We further illustrate the TCVAE applicability through multifaceted case studies and visualization in real-world scenarios.

Multi-Scale Transformer Pyramid Networks for Multivariate Time Series Forecasting

Multi-Scale Transformer Pyramid Networks for Multivariate Time Series Forecasting

DEformer: Dual Embedded Transformer for Multivariate Time Series Forecasting

DEformer: Dual Embedded Transformer for Multivariate Time Series Forecasting

Navigating Complexity: Harnessing AI for Multivariate Time Series Forecasting in Dynamic Environments

Multivariate Time Series Analysis (MTSA) plays a pivotal role in forecasting within diverse domains by addressing the complexities arising from interdependencies among multiple variables. This exploration delves into the fundamentals, methodologies, and applications of MTSA, elucidating its role in enhancing predictive capabilities. The key concepts in MTSA, including Vector Autoregression, Cointegration, Error Correction Models, and Granger Causality, form the foundation for understanding dynamic relationships among variables. The methodology section outlines the critical steps in MTSA, such as model specification, estimation, diagnostics, and forecasting. Additionally, the abstract explores the capabilities of Artificial Intelligence (AI) in time-series forecasting, emphasizing improved accuracy, long-term trend recognition, dynamic pattern recognition, and the handling of seasonality and anomalies. Specific AI models, such as Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM), Echo State Networks (ESNs), and Online Learning Algorithms, are discussed in detail, along with practical implementation examples. Furthermore, the abstract introduces the benefits and challenges associated with MTSA. The benefits include comprehensive insights, improved forecast accuracy, and real-world relevance, while challenges encompass data and model complexity, explicability, and the validity of assumptions. The discussion emphasizes the need for innovative approaches to explain the predictions of complex models and highlights ongoing research in developing explanability frameworks.

Multivariate Time-Series Forecasting Model: Predictability Analysis and Empirical Study

Multivariate time series forecasting has wide applications such as traffic flow prediction, supermarket commodity demand forecasting and etc., and a large number of forecasting models have been developed. Given these models, a natural question has been raised: what theoretical limits of forecasting accuracy can these models achieve? Recent works of urban human mobility prediction have made progress on the maximum predictability that any algorithm can achieve. However, existing approaches on maximum predictability on the multivariate time series fully ignore the interrelationship between multiple variables. In this paper, we propose a methodology to measure the upper limit of predictability for multivariate time series with multivariate constraint relations. The key of the proposed methodology is a novel entropy, named Multivariate Constraint Sample Entropy ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">McSE</i> ), to incorporate the multivariate constraint relations for better predictability. We conduct a systematic evaluation over eight datasets and compare existing methods with our proposed predictability and find that we get a higher predictability. We also find that the forecasting algorithms that capture the multivariate constraint relation information, such as GNN, can achieve higher accuracy, confirming the importance of multivariate constraint relations for predictability.

Weakly Guided Adaptation for Robust Time Series Forecasting

Robust multivariate time series forecasting is crucial in many cyberphysical and Internet of Things applications. Existing state-of-the-art robust forecasting models decompose time series into independent functions covering trends and periodicities. However, these independent functions fail to capture correlations among multiple time series, thereby reducing prediction accuracy. Moreover, existing robust forecasting models treat certain abrupt but normal changes, e.g., caused by holidays, as outliers because they occur infrequently and have data distributions that resemble those of outliers. This exacerbates model bias and reduces prediction accuracy. This paper aims to capture correlations across multiple time series and abrupt but normal changes, thereby improving prediction accuracy. We employ weak labels to partition the dataset into source and target domains. Then, we propose the Domain Adversarial Robust Forecaster (DARF). This forecasting model is based on adversarial domain adaptation and includes two novel modules: Correlated Robust Forecaster (CORF) and Domain Critic. Specifically, CORF constitutes an encoder-decoder framework proficient at robust multivariate time series forecasting, and Domain Critic works to reduce data bias. Extensive experiments and discussions show that DARF is capable of state-of-the-art forecasting accuracy.

A journey from univariate to multivariate functional time series: A comprehensive review

AbstractFunctional time series (FTS) analysis has emerged as a potent framework for modeling and forecasting time‐dependent data with functional attributes. In this comprehensive review, we navigate through the intricate landscape of FTS methodologies, meticulously surveying the core principles of univariate FTS and delving into the nuances of multivariate FTS. The journey commences with an exploration of the foundational aspects of univariate FTS analysis. We delve into representation, estimation, and modeling, spotlighting the effectiveness of various parametric and nonparametric models at our disposal. The stage then transitions to multivariate FTS analysis, where we confront the intricacies posed by high‐dimensional data. We explore strategies for dimensionality reduction, forecasting, and the integration of diverse parametric and nonparametric models within the multivariate realm. We also highlight commonly used R packages for modeling and forecasting FTS and multivariate FTS. Acknowledging the dynamic evolution of the field, we dissect challenges and chart future directions, paving a course for refinement and innovation. Through a fusion of multivariate statistics, functional analysis, and time series forecasting, this review underscores the interdisciplinary essence of FTS analysis. It not only reveals past accomplishments, but also illuminates the potential of FTS in unraveling insights and facilitating well‐informed decisions across diverse domains.This article is categorized under: Data: Types and Structure &gt; Time Series, Stochastic Processes, and Functional Data Statistical and Graphical Methods of Data Analysis &gt; Multivariate Analysis

Bayesian vector heterogeneous autoregressive modelling

The Bayesian vector autoregressive (BVAR) model with the Minnesota prior proposed by Litterman [Litterman RB. Forecasting with bayesian vector autoregressions-five years of experience. J Business Economic Statist.1986;4(1):25–38.] has been very successful in multivariate time series modelling, providing better forecasting performance. However, the conventional Minnesota prior for BVAR depends only on the latest lag; in turn, it is not suitable for multivariate long memory time series forecasting. This study extends BVAR to (possibly) high-dimensional long memory time series. To this end, we incorporate a vector heterogeneous autoregressive (VHAR) structure to accommodate long-term persistence and impose priors on distant lags as well. Our proposed Bayesian VHAR (BVHAR) models, the so-called BVHAR-S and BVHAR-L, are easy to implement by using the Normal-inverse-Wishart prior and added dummy variable approach of [Bańbura M, Giannone D, Reichlin L. Large bayesian vector auto regressions. J Appl Econom. 2010;25(1):71–92.]. A supporting simulation study also proves posterior consistency. We further apply our models to nine CBOE Volatility Indices (VIXs) and show that our BVHAR models perform best in forecasting with the narrowest forecasting region.

MvTS-library: An open library for deep multivariate time series forecasting

Modeling multivariate time series has been a subject for a long time, which attracts the attention of scholars from many fields including economics, finance, traffic, etc. As the number of models increases, it is desired to design a unified framework to implement and evaluate these models. Based on Pytorch, we propose MvTS, an open library for multivariate time series forecasting. Through a highly modular design, MvTS systematically integrates the various stages of the whole process of model training. Currently, the library contains 33 models and 23 datasets, and is available at https://github.com/MTS-BenchMark/MvTS. Based on MvTS, we conduct extensive experiments on public datasets and demonstrate that the models reproduced by MvTS are effective and universally applicable to many other datasets. MvTS is a systematic, comprehensive, extensible, and easy-to-use multivariate time series forecasting library, and we believe it will contribute to the research of multivariate time series to some extent.

Dynamic personalized graph neural network with linear complexity for multivariate time series forecasting

Spatio-temporal graph neural networks (GNN) that model inter-variable dependencies with learned graph structures for multivariate time series forecasting have received attention due to their superior performance. However, existing methods exhibit quadratic complexity, limiting the GNN’s capability to handle a large number of variables. Moreover, the message-passing mechanism employed by spatio-temporal GNNs can be seen as a type of Laplacian smoothing, which weakens the inherent evolution patterns of nodes and overlooks the dynamic impact of a node’s own information and that of its neighbors on future outcomes. In this paper, we propose a novel approach called the Dynamic Personalized Graph Neural Network (DPGNN), which presents a graph learning framework with linear complexity. To facilitate linearization, we conceptualize the spatio-temporal GNN as an information flow from the source (raw node features) to the sink (new node features). And the flow conservation principle is employed to generate the information attention mechanism to effectively model the dependency between variables. To address the second issue, inspired by PageRank, we design a dynamic personalized graph convolution scheme that explicitly quantifies the impact of each node on the subsequent results based on the dynamic graph signals at each propagation step. Experimental results on six benchmark datasets showcase the superiority of our approach over state-of-the-art methods, including both pre-defined graph structures and graph learning techniques.

Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

STTRE: A Spatio-Temporal Transformer with Relative Embeddings for multivariate time series forecasting

The prevalence of multivariate time series data across several disciplines fosters a demand and, subsequently, significant growth in the research and advancement of multivariate time series analysis. Drawing inspiration from a popular natural language processing model, the Transformer, we propose the Spatio-Temporal Transformer with Relative Embeddings (STTRE) to address multivariate time series forecasting. This work primarily focuses on developing a Transformer-based framework that can fully exploit the spatio-temporal nature of a multivariate time series by incorporating several of the Transformer’s key components, but with augmentations that allow them to excel in multivariate time series forecasting. Current Transformer-based models for multivariate time series often neglect the data’s spatial component(s) and utilize absolute position embeddings as their only means to detect the data’s temporal component(s), which we show is flawed for time series applications. The lack of emphasis on fully exploiting the spatio-temporality of the data can incur subpar results in terms of accuracy. We redesign relative position representations, which we rename to relative embeddings, to unveil a new method for detecting latent spatial, temporal, and spatio-temporal dependencies more effectively than previous Transformer-based models. We couple these relative embeddings with a restructuring of the Transformer’s primary sequence learning mechanism, multi-head attention, in a way that allows for full utilization of relative embeddings, thus achieving up to a 24% improvement in accuracy over other state-of-the-art multivariate time series models on a comprehensive selection of publicly available multivariate time series forecasting datasets.

Training LSTMS with circular-shift epochs for accurate event forecasting in imbalanced time series

The use of LSTM models for forecasting has increased substantially because they have proven to yield excellent performance in numerous applications involving time series consisting of normal observations. However, LSTM models tend to perform poorly in forecasting extreme events in imbalanced time series due to the lack of representative extreme event data during training. This study introduces a breakthrough Circular-Shift Epoch Training (CSET) strategy for LSTMs that makes the accurate forecasting of extreme events no more difficult than predicting events in normal time series with similar accuracies. CSET is (a) specifically designed to preserve the natural ordering of observations in epochs during training without any attempt to balance extreme and normal observations, (b) formulated for both univariate and multivariate time series forecasting, (c) applicable to scaler and vector forecasting for the multivariate case and can handle upward, downward, and mixed extremes without any modification, and (d) universal because it can be applied to train LSTMs to forecast events in normal time series or in imbalanced time series in exactly the same manner. Most importantly, the forecasting of normal time series and imbalanced time series do not have to be treated as separate problems. The results from an extensive set of experiments involving forecasting events in simulated and real imbalanced time series confirm that LSTM models trained using CSET dramatically outperform the standard fixed-epoch trained baseline models in (a) forecasting both extreme and normal events and (b) classifying extreme events. Given the ubiquitous use of LSTMs for forecasting, the accurate forecasting of extreme events through the incorporation of the innovative CSET strategy can have a major impact in decreasing the disastrous consequences of dangerous extreme events.