Multivariate Time Series Models Research Articles

PHM-Based Modeling for Cyberattack Classifier Performance

This research implements Prognostics and Health Management (PHM) using multiple linear regression and multivariate time series models to monitor and predict when the performance of a Machine Learning-based cyberattack classifier might degrade to an unacceptable level, enabling preemptive maintenance strategies.

Displacement prediction of tunnel entrance slope based on LSSVM and bacterial foraging optimization algorithm

In order to realize the effective prediction of landslide risk in the tunnel entrance area, an multivariate time series model is established on the basis of the traditional model, taking temperature and rainfall factors as additional input indicators. Bacterial foraging optimization algorithm (BFOA) is used to search the global optimal solution of the key parameters \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\gamma$$\\end{document} and \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sigma^{2}$$\\end{document} of least squares support vector machine (LSSVM) to improve its regression accuracy, and the evolved LSSVM is used to describe the aforementioned multivariate time series model. At the same time, a remote real-time internet of things (IoT) monitoring system for the tunnel entrance section, including monitoring indicators such as surface subsidence, temperature, and rainfall, has also been designed and implemented, providing a stable and accurate data source for the realization of this prediction model. Based on the engineering measurement data, the accuracy of the established model is checked and analyzed, the optimal value of historical data amount is determined to be 5 days, and the optimal value of prediction step is 1 day. The research results are applied in the construction of Wendong tunnel of Molin expressway, Yunnan, China. Practice shows that the prediction results of the multivariate time series model established in this study is accurate. This method can realize the prediction and early warning of slope risk, which provides a effective technical means for risk control of tunnel portal section.

Multivariate stochastic generation of meteorological data for building simulation through interdependent meteorological processes

In recent years, the uncertainty of weather conditions and the impact of future climate change on building energy assessment has received increasing attention. As an important part of these studies, several types of methods for generating stochastic meteorological data have also been developed. Since solar radiation drops to zero at night, unlike the continuous 24-hour data for elements such as temperature and humidity, this has posed challenges for previous research to fully account for the simultaneity among multiple elements. Therefore, this study proposes a framework for meteorological data generation: First, perform multivariate time series modeling of meteorological data of air temperature, solar radiation and absolute humidity at 12:00 of each day of a typical year based on the S-vine copula method and simulating daily series data at 12:00 for 365 days. Then, based on the probability of change of each element evaluated from the historical meteorological observation data, the daily series data at 12:00 were expanded to 24 h, after which the yearly stochastic weather data were obtained. The analysis of 30 years of stochastic data generated by this method, compared with the original data, reveals that air temperature and solar radiation closely match the original distribution characteristics, except for a minor deviation in the absolute humidity’s kurtosis. Furthermore, the comparison of thermal load distributions for office buildings shows that the original data curve falls within the range of the generated data. This suggests that the generated data includes more information about uncertainty but still keeps the original data’s characteristics.

A parallel recursive framework for modelling time series

Abstract Time series modelling is of significance to several scientific fields. Several approaches based on statistics, machine learning or combinations have been utilized. In order to model and forecast time series a novel parallel framework based on recursive pseudoinverse matrices is proposed. This framework enables the design of arbitrary statistical and machine learning models, adaptively, from a set of potential basis functions. This unification enables compact definition of existing and new models as well as easy implementation for new massively parallel architectures. The choice of appropriate basis functions is analysed and the fitting accuracy, termination criteria and model update operations are presented. A block variant for multivariate time series is also proposed. Parallel GPU implementation and performance optimization of the framework are provided, based on mixed precision arithmetic and matrix operations. The use of different basis functions is showcased with respect to various model univariate and multivariate time series for applications such as regression, frequency estimation and automatic trend detection. Discussions on limitations and future directions of research are also provided.

Evolution of COVID-19 dynamics in Guangdong Province, China: an endemic-epidemic modeling study

BackgroundFrom January 2020 to June 2022, strict interventions against COVID-19 were implemented in Guangdong Province, China. However, the evolution of COVID-19 dynamics remained unclear in this period.ObjectivesThis study aims to investigate the evolution of within- and between-city COVID-19 dynamics in Guangdong, specifically during the implementation of rigorous prevention and control measures. The intent is to glean valuable lessons that can be applied to refine and optimize targeted interventions for future crises.MethodsData of COVID-19 cases and synchronous interventions from January 2020 to June 2022 in Guangdong Province were collected. The epidemiological characteristics were described, and the effective reproduction number (Rt) was estimated using a sequential Bayesian method. Endemic-epidemic multivariate time-series model was employed to quantitatively analyze the spatiotemporal component values and variations, to identify the evolution of within- and between-city COVID-19 dynamics.ResultsThe incidence of COVID-19 in Guangdong Province was 12.6/100,000 population (15,989 cases) from January 2020 to June 2022. The Rt predominantly remained below 1 and increased to a peak of 1.39 in Stage 5. As for the evolution of variations during the study period, there were more spatiotemporal components in stage 1 and 5. All components were fewer from Stage 2 to Stage 4. Results from the endemic-epidemic multivariate time-series model revealed a strong follow-up impact from previous infections in Dongguan, Guangzhou and Zhanjiang, with autoregressive components of 0.48, 0.45 and 0.36, respectively. Local risk was relatively high in Yunfu, Shanwei and Shenzhen, with endemic components of 1.17, 1.04 and 0.71, respectively. The impact of the epidemic on the neighboring regions was significant in Zhanjiang, Shenzhen and Zhuhai, with epidemic components of 2.14, 1.92, and 1.89, respectively.ConclusionThe findings indicate the presence of spatiotemporal variation of COVID-19 in Guangdong Province, even with the implementation of strict interventions. It’s significant to prevent transmissions within cities with dense population. Preventing spatial transmissions between cities is necessary when the epidemic is severe. To better cope with future crises, interventions including vaccination, medical resource allocation and coordinated non-pharmaceutical interventions were suggested.

Applying the Expectation-Maximization Algorithm to Multivariate Signal Extraction

Understanding time series signal extraction is vital for federal agencies, especially as data collection accelerates. One natural extension is moving from univariate to multivariate signal extraction, which offers the promise of reducing extraction error by exploiting cross-sectional relationships. However, such an extension ushers in new computational challenges, viz. larger parameter spaces and more complicated objective functions. The Expectation-Maximization (EM) algorithm provides a methodology to implicitly compute (or approximate) maximum likelihood estimates (MLEs). This paper provides methodology for applying the EM algorithm to a class of latent component multivariate time series models that allow for a nuanced specification of the unobserved signal. We derive an explicit formula for the maximization step, which facilitates computation speed while also improving the stability of the algorithm. Numerical studies demonstrate EM’s ability to efficiently compute MLEs in low-dimensional systems, while also providing feasible estimates in moderate-dimensional systems where MLEs are infeasible to compute. Applications to monthly housing starts and daily immigration are provided.

VTNet: A multi-domain information fusion model for long-term multi-variate time series forecasting with application in irrigation water level

Time series forecasting is intricately tied to production and life, garnering widespread attention over an extended period. Enhancing the performance of long-term multivariate time series forecasting (MTSF) poses a highly challenging task, as it requires mining complicated and obscure temporal patterns in many aspects. For this reason, this paper proposes a long-term forecasting model based on multi-domain fusion (VTNet) to adaptively capture and refine multi-scale intra- and inter-variate dependencies. In contrast to previous techniques, we devise a dual-stream learning architecture. Firstly, the fast Fourier Transform (FFT) is adopted to extract frequency domain information. The original sequences are then transformed into 2D visual features in the temporal-frequency domain, and a 2D-TBlock is designed for multi-scale dynamic learning. Secondly, a combination of convolution and recurrent networks continues to explore the local temporal features and preserve the global trend. Finally, multi-modal circulant fusion is applied to achieve a more comprehensive and enriched feature fusion representation, further promoting overall performance. Extensive experiments are conducted on 9 public benchmark datasets and the real-world irrigation water level to showcase VTNet’s promoted performance and generalization. Moreover, VTNet yields 46.93% and 25.36% relative improvements for water level forecasting, revealing its potential application value in water-saving planning and extreme event early warning.

Scaled envelope models for multivariate time series

Vector autoregressive (VAR) models have become a popular choice for modeling multivariate time series data due to their simplicity and ease of use. Efficient estimation of VAR coefficients is an important problem. The envelope technique for VAR models is demonstrated to have the potential to yield significant gains in efficiency and accuracy by incorporating linear combinations of the response vector that are essentially immaterial to the estimation of the VAR coefficients. However, inferences based on envelope VAR (EVAR) models are not invariant or equivariant upon the rescaling of the VAR responses, limiting their application to time series data that are measured in the same or similar units. In scenarios where VAR responses are measured on different scales, the efficiency improvements promised by envelopes are not always guaranteed. To address this limitation, we introduce the scaled envelope VAR (SEVAR) model, which preserves the efficiency-boosting capabilities of standard envelope techniques while remaining invariant to scale changes. The asymptotic characteristics of the proposed estimators are established based on different error assumptions. Simulation studies and real-data analysis are conducted to demonstrate the efficiency and effectiveness of the proposed model. The numerical results corroborate our theoretical findings.

Latent level correlation modeling of multivariate discrete-valued financial time series

Latent level correlation modeling of multivariate discrete-valued financial time series

Deep Learning-Based Prediction of Daily COVID-19 Cases Using X (Twitter) Data.

Due to the importance of COVID-19 control, innovative methods for predicting cases using social network data are increasingly under attention. This study aims to predict confirmed COVID-19 cases using X (Twitter) social network data (tweets) and deep learning methods. We prepare data extracted from tweets by natural language processing (NLP) and consider the daily G-value (growth rate) as the target variable of COVID-19, collected from the worldometer. We develop and evaluate a time series mixer (TSMixer) predictive model for multivariate time series. The mean squared error (MSE) loss on the test dataset was 0.0063 for 24-month Gvalue prediction when using the MinMax normalization with recursive feature elimination (RFE) and average or min aggregation method. Our findings illuminate the potential of integrating social media data to enhance daily COVID-19 case predictions and are applicable also for epidemiological forecasting purposes.

An information-theoretic framework for conditional causality analysis of brain networks.

Identifying directed network models for multivariate time series is a ubiquitous problem in data science. Granger causality measure (GCM) and conditional GCM (cGCM) are widely used methods for identifying directed connections between time series. Both GCM and cGCM have frequency-domain formulations to characterize the dependence of time series in the spectral domain. However, the original methods were developed using a heuristic approach without rigorous theoretical explanations. To overcome the limitation, the minimum-entropy (ME) estimation approach was introduced in our previous work (Ning & Rathi, 2018) to generalize GCM and cGCM with more rigorous frequency-domain formulations. In this work, this information-theoretic framework is further generalized with three formulations for conditional causality analysis using techniques in control theory, such as state-space representations and spectral factorizations. The three conditional causal measures are developed based on different ME estimation procedures that are motivated by equivalent formulations of the classical minimum mean squared error estimation method. The relationship between the three formulations of conditional causality measures is analyzed theoretically. Their performance is evaluated using simulations and real neuroimaging data to analyze brain networks. The results show that the proposed methods provide more accurate network structures than the original approach.

A novel method to select time-varying multivariate time series models for the surveillance of infectious diseases

BackgroundDescribing the transmission dynamics of infectious diseases across different regions is crucial for effective disease surveillance. The multivariate time series (MTS) model has been widely adopted for constructing cross-regional infectious disease transmission networks due to its strengths in interpretability and predictive performance. Nevertheless, the assumption of constant parameters frequently disregards the dynamic shifts in disease transmission rates, thereby compromising the accuracy of early warnings. This study investigated the applicability of time-varying MTS models in multi-regional infectious disease monitoring and explored strategies for model selection.MethodsThis study focused on two prominent time-varying MTS models: the time-varying parameter-stochastic volatility-vector autoregression (TVP-SV-VAR) model and the time-varying VAR model using the generalized additive framework (tvvarGAM), and intended to explore and verify their applicable conditions for the surveillance of infectious diseases. For the first time, this study proposed the time delay coefficient and spatial sparsity indicators for model selection. These indicators quantify the temporal lags and spatial distribution of infectious disease data, respectively. Simulation study adopted from real-world infectious disease surveillance was carried out to compare model performances under various scenarios of spatio-temporal variation as well as random volatility. Meanwhile, we illustrated how the modelling process could help the surveillance of infectious diseases with an application to the influenza-like case in Sichuan Province, China.ResultsWhen the spatio-temporal variation was small (time delay coefficient: 0.1–0.2, spatial sparsity:0.1–0.3), the TVP-SV-VAR model was superior with smaller fitting residuals and standard errors of parameter estimation than those of the tvvarGAM model. In contrast, the tvvarGAM model was preferable when the spatio-temporal variation increased (time delay coefficient: 0.2–0.3, spatial sparsity: 0.6–0.9).ConclusionThis study emphasized the importance of considering spatio-temporal variations when selecting appropriate models for infectious disease surveillance. By incorporating our novel indicators—the time delay coefficient and spatial sparsity—into the model selection process, the study could enhance the accuracy and effectiveness of infectious disease monitoring efforts. This approach was not only valuable in the context of this study, but also has broader implications for improving time-varying MTS analyses in various applications.

Causal analysis among azimuthal Fourier modes in linear plasma based on multivariate time series models

This study investigates the causal relationships among azimuthal Fourier modes in linear plasma turbulence using multivariate time series models. We elucidate the dynamics of mode interactions in magnetized plasmas by employing the vector autoregressive model and Granger causality analysis. Our analysis, based on data from the plasma assembly for nonlinear turbulence analysis, reveals significant variations in causality with changing pressure conditions. Modes form weakly coupled clusters at lower pressures, while higher pressures lead to stronger coupling and larger clusters. The impulse response function further provides insights into the temporal propagation and nature of influences between modes. These findings enhance the understanding of spatial pattern formation in magnetized plasmas and offer a quantitative framework for analyzing plasma turbulence dynamics.

An anomaly detection model for multivariate time series with anomaly perception

Multivariate time series anomaly detection is a crucial data mining technique with a wide range of applications in areas such as IT applications. Currently, the majority of anomaly detection methods for time series data rely on unsupervised approaches due to the rarity of anomaly labels. However, in real-world scenarios, obtaining a limited number of anomaly labels is feasible and affordable. Effective usage of these labels can offer valuable insights into the temporal characteristics of anomalies and play a pivotal role in guiding anomaly detection efforts. To improve the performance of multivariate time series anomaly detection, we proposed a novel deep learning model named EDD (Encoder-Decoder-Discriminator) that leverages limited anomaly samples. The EDD model innovatively integrates a graph attention network with long short term memory (LSTM) to extract spatial and temporal features from multivariate time series data. This integrated approach enables the model to capture complex patterns and dependencies within the data. Additionally, the model skillfully maps series data into a latent space, utilizing a carefully crafted loss function to cluster normal data tightly in the latent space while dispersing abnormal data randomly. This innovative design results in distinct probability distributions for normal and abnormal data in the latent space, enabling precise identification of anomalous data. To evaluate the performance of our EDD model, we conducted extensive experimental validation across three diverse datasets. The results demonstrate the significant superiority of our model in multivariate time series anomaly detection. Specifically, the average F1-Score of our model outperformed the second-best method by 2.7% and 73.4% in both evaluation approaches, respectively, highlighting its superior detection capabilities. These findings validate the effectiveness of our proposed EDD model in leveraging limited anomaly samples for accurate and robust anomaly detection in multivariate time series data.

Short-Term Power Load Forecasting Method Based on GRU-Transformer Combined Neural Network Model

Load Forecast (LF) is an important task in the planning, control and application of public power systems. Accurate Short Term Load Forecast (STLF) is the premise of safe and economical operation of a power system. In the research of short-term power load forecasting, machine learning and deep learning are the most popular methods at present, but there still exists a problem that the single and simple structure of power load forecasting model leads to low accuracy of load forecasting. In order to improve the accuracy of STLF, a Gated Cycle Unit (GRU)-Transformer combined neural network model is proposed. Transformer encoder structure is used as feature extractor to mine the complex mapping relationships between the input features and load. The advantage of self-attention mechanism is used to solve the problem of information loss of long sequences in short-term power load forecasting. At the same time, the multivariate time series model of GRU is used for model training. The experimental results on the power load data set of a certain region in southwest China and Panama City show that the proposed combined model prediction method has higher accuracy than those proposed in other literatures, which further proves its feasibility and superiority.

Multiview Spatial-Temporal Meta-Learning for Multivariate Time Series Forecasting.

Multivariate time series modeling has been essential in sensor-based data mining tasks. However, capturing complex dynamics caused by intra-variable (temporal) and inter-variable (spatial) relationships while simultaneously taking into account evolving data distributions is a non-trivial task, which faces accumulated computational overhead and multiple temporal patterns or distribution modes. Most existing methods focus on the former direction without adaptive task-specific learning ability. To this end, we developed a holistic spatial-temporal meta-learning probabilistic inference framework, entitled ST-MeLaPI, for the efficient and versatile learning of complex dynamics. Specifically, first, a multivariate relationship recognition module is utilized to learn task-specific inter-variable dependencies. Then, a multiview meta-learning and probabilistic inference strategy was designed to learn shared parameters while enabling the fast and flexible learning of task-specific parameters for different batches. At the core are spatial dependency-oriented and temporal pattern-oriented meta-learning approximate probabilistic inference modules, which can quickly adapt to changing environments via stochastic neurons at each timestamp. Finally, a gated aggregation scheme is leveraged to realize appropriate information selection for the generative style prediction. We benchmarked our approach against state-of-the-art methods with real-world data. The experimental results demonstrate the superiority of our approach over the baselines.

A Bayesian non-stationary heteroskedastic time series model for multivariate critical care data.

We propose a multivariate GARCH model for non-stationary health time series by modifying the observation-level variance of the standard state space model. The proposed model provides an intuitive and novel way of dealing with heteroskedastic data using the conditional nature of state-space models. We follow the Bayesian paradigm to perform the inference procedure. In particular, we use Markov chain Monte Carlo methods to obtain samples from the resultant posterior distribution. We use the forward filtering backward sampling algorithm to efficiently obtain samples from the posterior distribution of the latent state. The proposed model also handles missing data in a fully Bayesian fashion. We validate our model on synthetic data and analyze a data set obtained from an intensive care unit in a Montreal hospital and the MIMIC dataset. We further show that our proposed models offer better performance, in terms of WAIC than standard state space models. The proposed model provides a new way to model multivariate heteroskedastic non-stationary time series data. Model comparison can then be easily performed using the WAIC.

Accurate short-term GHI forecasting using a novel temporal convolutional network model

Global energy demand is on the rise, driven by factors such as population growth and economic development.Utilizing renewable energy is crucial to meeting this energy demand in light of global warming and the depletion of fossil resources.One of the renewable energy sources that is extensively utilized in multiple countries worldwide is photovoltaic energy.While integrating photovoltaic (PV) energy into the grid offers substantial economic and environmental advantages, its intermittent nature presents challenges to maintaining grid stability at high penetration levels. Accurate solar energy estimations are essential for photovoltaic (PV) based energy facilities to enable early participation in energy markets and efficient resource planning. This study proposes and evaluates a Temporal Convolutional Network (TCN) model that can rapidly predict global horizontal irradiance (GHI) using univariate and multivariate approaches. To our knowledge, this is the first study to employ a TCN model for GHI forecasting. The univariate model solely considers GHI, while the multivariate time series models incorporate a combination of six variables: global horizontal irradiance, active power of a module plant, PV module temperature, wind speed, air temperature, and air pressure. The proposed model’s effectiveness is validated by comparing its performance to benchmark models, including MLP, RNN, GRU, and LSTM. The prediction models’ performance is evaluated using root mean square error (RMSE), coefficient of determination (R2), and mean absolute error (MAE). The results demonstrate that the effectiveness of univariate TCN models is evident in their performance for 5, 10, 15, and 20 min ahead short-term GHI forecasting, its mean absolute error (MAE) values achieved are 22.935 W/m2 33.47 W/m2, 36.9994 W/m2, and 41.9946 W/m2, respectively. The multivariate model, incorporating additional variables, yielded higher MAE values : 37.928 W/m2, 55.88 W/m2, 66.61 W/m2, and 74.82 W/m2, respectively. These results suggest that the TCN model’s capability to capture both long-term and short-term dependencies within the data contributes to its accuracy for GHI forecasting compared to other models.

Multivariate Bayesian Time-Series Model with Multi-temporal Convolution Network for Forecasting Stock Market During COVID-19 Pandemic

The paper proposes a hybrid algorithm for forecasting multiple correlated time-series data, which consists of two main steps. First, it employs a multivariate Bayesian structural time series (MBSTS) approach as a base step. This method allows for the incorporation of potentially high-dimensional regression components, and it utilizes spike and slab priors to identify a parsimonious model. Second, the algorithm includes a post-model fitting diagnostic step where the residuals from the MBSTS step are processed through a multi-input/output temporal convolutional network (M-TCN) with multiple time scale feature learning. This step serves as an alternative to traditional subjective residual-based diagnostic procedures in time-series analysis, with the aim of improving forecasting accuracy. The key advantage of the M-TCN is its ability to capture sequential information efficiently. The M-TCN expands the field of convolution kernel without increasing the number of parameters, thus enhancing the capacity of model to capture complex sequential patterns. The paper presents two applications showcasing the effectiveness of the proposed hybrid algorithm. First, it utilizes pre-lockdown data from eleven Nifty stock sectoral indices to predict stock price movements, including the initial post-lockdown upturn. In the second application, it focuses on stock market data from pharmaceutical companies involved in manufacturing COVID-19 vaccines. In both cases, sentiment data sourced from newspapers and social media serve as the regression component. Through rigorous analysis, the paper demonstrates that the hybrid model outperforms various benchmark models, including LSTM, Bidirectional Encoder Representations from Transformers (BERT)-based LSTM, Deep Transformer Model, and GRU, among others, in terms of forecasting accuracy. This underscores the utility of the hybrid algorithm, particularly in predicting stock market trends during the COVID-19 pandemic period and its associated market dynamics.

Negative binomial community network vector autoregression for multivariate integer-valued time series

Modeling multivariate integer-valued time series with appropriate methods is currently a popular research topic. In this paper, we propose a multivariate integer-valued autoregressive time series model based on a fixed network community structure. We use the negative binomial distribution as the conditional marginal distribution and a copula to construct the conditional joint distribution. The newly proposed model introduces the heterogeneity of nodes. Stability conditions are provided for both fixed and increasing dimensions. We estimate the parameters of the proposed model by maximizing the quasi-likelihood function with known and unknown community membership matrices, respectively. Corresponding asymptotic properties of parameter estimates are also provided. A simulation study is conducted to demonstrate the asymptotic behavior of the proposed model, and two real datasets are employed to compare the proposed model with other competitive models.