Vector Autoregressive Process Research Articles

Conditional independence graphs for multivariate autoregressive models by convex optimization: Efficient algorithms

In this paper, we introduce novel algorithms for inferring the conditional independence graph of a vector autoregressive (VAR) process. As part of this work, we derive the renormalized maximum likelihood criterion for VAR-order selection and prove its consistency. Finding the graphical model for VAR reduces to identify the sparsity pattern of the inverse of its spectral density matrix; we show how efficient implementations of convex optimization algorithms can be used to solve this problem; in our approach, the high-sparsity assumption is not needed. We conduct experiments with simulated data, air pollution data and stock market data for demonstrating that our algorithms are faster and more accurate than similar methods proposed in the previous literature.

Sparse vector Markov switching autoregressive models. Application to multivariate time series of temperature

Multivariate time series are of interest in many fields including economics and environment. The dynamical processes occurring in these domains often exhibit a mixture of different dynamics so that it is common to describe them using Markov Switching vector autoregressive processes. However the estimation of such models is difficult even when the dimension is not so high because of the number of parameters involved. A Smoothly Clipped Absolute Deviation penalization of the likelihood is proposed to shrink the parameters towards zeros and regularize the inference problem which is generally ill-posed. The Expectation Maximization algorithm built for maximizing the penalized likelihood is described in detail and tested on simulated data and real data consisting of daily mean temperature.

Monitoring multistage processes with autocorrelated observations

In multistage manufacturing processes, autocorrelations within stages over time are prevalent and the classical control charts are often ineffective in monitoring such processes. In this paper, we derive a linear state space model of an autocorrelated multistage process as a vector autoregressive process, and construct novel multivariate control charts, CBAM and Conditional-based MEWMA, for detecting the mean changes in a multistage process based on a projection scheme by incorporating in-control stage information. When in-control stages are unknown, finding in-control stages is a challenging issue due to the autocorrelations over time and the sequential correlations between stages. To overcome this difficulty, we propose a conditional-based selection that chooses stages with strong evidences of in-control stage using the cascading property of multistage processes. The information of selected stages is effectively utilised in obtaining powerful test statistics for detecting a mean change. The performance of the proposed charts is compared with other existing procedures under different scenarios. Both simulation studies and a real example show the effectiveness of the conditional-based charts in detecting a wide range of small mean shifts compared with the other existing control charts.

High-dimensional and banded vector autoregressions

We consider a class of vector autoregressive models with banded coefficient matrices. The setting represents a type of sparse structure for high-dimensional time series, though the implied autocovariance matrices are not banded. The structure is also practically meaningful when the order of component time series is arranged appropriately. The convergence rates for the estimated banded autoregressive coefficient matrices are established. We also propose a Bayesian information criterion for determining the width of the bands in the coefficient matrices, which is proved to be consistent. By exploring some approximate banded structure for the autocovariance functions of banded vector autoregressive processes, consistent estimators for the auto-covariance matrices are constructed.

Modeling and forecasting multivariate electricity price spikes

We consider the problem of forecasting the occurrence of extreme prices in the Australian electricity markets from high frequency spot prices. In particular, we are interested in the simultaneous occurrence of such so-called spikes in two or more markets. Our approach is based on a novel dynamic model for multivariate binary outcomes, which allows the latent variables driving these observed outcomes to follow a vector autoregressive process. Furthermore the model is constructed using a copula representation for the joint distribution of the resulting innovations. This has several advantages over the standard multivariate probit model. First, it allows for nonlinear dependence between the error terms. Second, the distribution of the latent errors can be chosen freely. Third, the computational burden can be greatly reduced making estimation feasible in higher dimensions and for large samples. The model is applied to spikes in half-hourly electricity prices in four interconnected Australian markets. The multivariate model provides a superior fit compared to single-equation models and generates better forecasts for spike probabilities. Furthermore, evidence of spillover dynamics between the markets is revealed.

Statistical wind direction modeling for the analysis of large scale wind power generation

AbstractUnderstanding the effects of large‐scale wind power generation on the electric power system is growing in importance as the amount of installed generation increases. In addition to wind speed, the direction of the wind is important when considering wind farms, as the aggregate generation of the farm depends on the direction of the wind. This paper introduces the wrapped Gaussian vector autoregressive process for the statistical modeling of wind directions in multiple locations. The model is estimated using measured wind direction data from Finland. The presented methodology can be used to model new locations without wind direction measurements. This capability is tested with two locations that were left out of the estimation procedure. Through long‐term Monte Carlo simulations, the methodology is used to analyze two large‐scale wind power scenarios with different geographical distributions of installed generation. Wind generation data are simulated for each wind farm using wind direction and wind speed simulations and technical wind farm information. It is shown that, compared with only using wind speed data in simulations, the inclusion of simulated wind directions enables a more detailed analysis of the aggregate wind generation probability distribution. Copyright © 2016 John Wiley & Sons, Ltd.

A theoretical stochastic control framework for adapting radiotherapy to hypoxia

Hypoxia, that is, insufficient oxygen partial pressure, is a known cause of reduced radiosensitivity in solid tumors, and especially in head-and-neck tumors. It is thus believed to adversely affect the outcome of fractionated radiotherapy. Oxygen partial pressure varies spatially and temporally over the treatment course and exhibits inter-patient and intra-tumor variation. Emerging advances in non-invasive functional imaging offer the future possibility of adapting radiotherapy plans to this uncertain spatiotemporal evolution of hypoxia over the treatment course.We study the potential benefits of such adaptive planning via a theoretical stochastic control framework using computer-simulated evolution of hypoxia on computer-generated test cases in head-and-neck cancer. The exact solution of the resulting control problem is computationally intractable. We develop an approximation algorithm, called certainty equivalent control, that calls for the solution of a sequence of convex programs over the treatment course; dose–volume constraints are handled using a simple constraint generation method. These convex programs are solved using an interior point algorithm with a logarithmic barrier via Newton’s method and backtracking line search. Convexity of various formulations in this paper is guaranteed by a sufficient condition on radiobiological tumor-response parameters. This condition is expected to hold for head-and-neck tumors and for other similarly responding tumors where the linear dose–response parameter is larger than the quadratic dose–response parameter.We perform numerical experiments on four test cases by using a first-order vector autoregressive process with exponential and rational-quadratic covariance functions from the spatiotemporal statistics literature to simulate the evolution of hypoxia. Our results suggest that dynamic planning could lead to a considerable improvement in the number of tumor cells remaining at the end of the treatment course. Through these simulations, we also gain insights into when and why dynamic planning is likely to yield the largest benefits.

Estimation and Inference of FAVAR Models

The factor-augmented vector autoregressive (FAVAR) model is now widely used in macroeconomics and finance. In this model, observable and unobservable factors jointly follow a vector autoregressive process, which further drives the comovement of a large number of observable variables. We study the identification restrictions for FAVAR models, and propose a likelihood-based two-step method to estimate the model. The estimation explicitly accounts for factors being partially observed. We then provide an inferential theory for the estimated factors, factor loadings, and the dynamic parameters in the VAR process. We show how and why the limiting distributions are different from the existing results. Supplementary materials for this article are available online.

A Short-Term Nodal Voltage Phasor Forecasting Method Using Temporal and Spatial Correlation

This paper presents a new method for short-term nodal voltage phasor forecasting in power systems to which a large number of renewable resources and microgrids are connected. Its motivation stems from the observation that such systems are characterized by an unconventional topology along with the presence of a number of buses with bursty power injection patterns, which in turn significantly impacts the temporal and spatial correlation of the nodal voltage phasors. Simulation results carried out on three IEEE test systems reveal that this new pattern affects the number of dominant buses, termed electrical hubs, and, subsequently, the time and spatial correlation among nodal voltage angles. They also reveal that the nodal voltage magnitudes exhibit time correlation, but no spatial correlation. In this paper, time series are represented by autoregressive (AR) processes while multivariate time series with spatial correlation are represented by vector autoregressive (VAR) processes. Statistical tests show that an order one for both models is adequate. The non-zero parameters of the VAR(1) models are identified using metrics on electrical connectivity, centrality, and node significance. The good performance of the proposed method is demonstrated on the IEEE 57-, 118-, and 300-bus test systems in presence of large-scale distributed wind farms and microgrids, and under loss of system observability.

Multiscale analysis of information dynamics for linear multivariate processes.

In the study of complex physical and physiological systems represented by multivariate time series, an issue of great interest is the description of the system dynamics over a range of different temporal scales. While information-theoretic approaches to the multiscale analysis of complex dynamics are being increasingly used, the theoretical properties of the applied measures are poorly understood. This study introduces for the first time a framework for the analytical computation of information dynamics for linear multivariate stochastic processes explored at different time scales. After showing that the multiscale processing of a vector autoregressive (VAR) process introduces a moving average (MA) component, we describe how to represent the resulting VARMA process using statespace (SS) models and how to exploit the SS model parameters to compute analytical measures of information storage and information transfer for the original and rescaled processes. The framework is then used to quantify multiscale information dynamics for simulated unidirectionally and bidirectionally coupled VAR processes, showing that rescaling may lead to insightful patterns of information storage and transfer but also to potentially misleading behaviors.

Relationship between energy consumption and economic growth in South Africa: Evidence from the bootstrap rolling-window approach

ABSTRACTThis article examines the relationship between energy consumption (EC) and economic growth for South Africa for the period 1971–2009. Most studies examining this relationship do assume that it remains constant through the years; however, the reality might be different since many factors can affect the existence and direction of this causality. This article looks at a bivariate vector autoregressive process and takes into consideration any instability in the model using bootstrap rolling Granger non-causality tests. Full-sample Granger causality tests report no causal relationship between the two variables. Moreover, parameter stability tests detect instability in the model which means that the full-sample Granger causality results are not valid. We therefore allow for the possibility of structural breaks by using bootstrap rolling-window Granger causality tests. Although our results are not very strong, we do however find a sub-period from 1987 to 1989 where EC has a causal effect on gross domestic product growth. Except for this brief sub-period, the results show no linkage between economic growth and EC.

Marginal distribution of Markov-switching VAR processes

ABSTRACTWe make available simple and accurate closed-form approximations to the marginal distribution of Markov-switching vector auto-regressive (MS VAR) processes. The approximation is built upon the property of MS VAR processes of being Gaussian conditionally on any semi-infinite sequence of the latent state. Truncating the semi-infinite sequence and averaging over all possible sequences of that finite length yields a mixture of normals that converges to the unknown marginal distribution as the sequence length increases. Numerical experiments confirm the viability of the approach which extends to the closely related class of MS state space models. Several applications are discussed.

Convolutional autoregressive models for functional time series

Functional data analysis has became an increasingly popular class of problems in statistical research. However, functional data observed over time with serial dependence remains a less studied area. Motivated by Bosq (2000), who first introduced the functional autoregressive models, we propose a convolutional functional autoregressive model, where the function at time t is a result of the sum of convolutions of the past functions and a set of convolution functions, plus a noise process, mimicking the vector autoregressive process. It provides an intuitive and direct interpretation of the dynamics of a stochastic process. Instead of principal component analysis commonly used in functional data analysis, we adopt a sieve estimation procedure based on B-spline approximation of the convolution functions. We establish convergence rate of the proposed estimator, and investigate its theoretical properties. The model building, model validation, and prediction procedures are also developed. Both simulated and real data examples are presented.

Tests of the co-integration rank in VAR models in the presence of a possible break in trend at an unknown point

In this paper we consider the problem of testing for the co-integration rank of a vector autoregressive process in the case where a trend break may potentially be present in the data. It is known that un-modelled trend breaks can result in tests which are incorrectly sized under the null hypothesis and inconsistent under the alternative hypothesis. Extant procedures in this literature have attempted to solve this inference problem but require the practitioner to either assume that the trend break date is known or to assume that any trend break cannot occur under the co-integration rank null hypothesis being tested. These procedures also assume the autoregressive lag length is known to the practitioner. All of these assumptions would seem unreasonable in practice. Moreover in each of these strands of the literature there is also a presumption in calculating the tests that a trend break is known to have happened. This can lead to a substantial loss in finite sample power in the case where a trend break does not in fact occur. Using information criteria based methods to select both the autoregressive lag order and to choose between the trend break and no trend break models, using a consistent estimate of the break fraction in the context of the former, we develop a number of procedures which deliver asymptotically correctly sized and consistent tests of the co-integration rank regardless of whether a trend break is present in the data or not. By selecting the no break model when no trend break is present, these procedures also avoid the potentially large power losses associated with the extant procedures in such cases.

Wind speed modeling using a vector autoregressive process with a time-dependent intercept term

Abstract The effect of wind power on the electric power system is becoming increasingly important as more wind power is installed. This paper presents a Vector-Autoregressive-To-Anything (VARTA) process with a time-dependent intercept to model wind speeds in multiple locations. The model considers both temporal and spatial dependency structures in detail. It can be used in Monte Carlo simulations to assess the risk of very high or low wind speeds occurring in multiple locations at the same time and during consecutive hours. The long term simulation and short term forecasting results are assessed using measurements from 21 locations in Finland. A discussion on the applicability of the presented wind speed simulation method for the risk assessment of power systems is given.

Filtering, Prediction and Simulation Methods for Noncausal Processes

This article discusses filtering, prediction and simulation in univariate and multivariate noncausal processes. A closed‐form functional estimator of the predictive density for noncausal and mixed processes is introduced that provides prediction intervals up to a finite horizon H. A state‐space representation of a noncausal and mixed multivariate vector autoregressive process is derived in two ways‐by the partial fraction decomposition or from the real Jordan canonical form. A recursive BHHH algorithm for the maximization of the approximate log‐likelihood function is proposed, which calculates the filtered values of the unobserved causal and noncausal components of the process. The new methods are illustrated by a simulation study involving a univariate noncausal process with infinite variance.

Reconstruction of late Holocene climate based on tree growth and mechanistic hierarchical models

Reconstruction of pre‐instrumental, late Holocene climate is important for understanding how climate has changed in the past and how climate might change in the future. Statistical prediction of paleoclimate from tree ring widths is challenging because tree ring widths are a one‐dimensional summary of annual growth that represents a multi‐dimensional set of climatic and biotic influences. We develop a Bayesian hierarchical framework using a nonlinear, biologically motivated tree ring growth model to jointly reconstruct temperature and precipitation in the Hudson Valley, New York. Using a common growth function to describe the response of a tree to climate, we allow for species‐specific parameterizations of the growth response. To enable predictive backcasts, we model the climate variables with a vector autoregressive process on an annual timescale coupled with a multivariate conditional autoregressive process that accounts for temporal correlation and cross‐correlation between temperature and precipitation on a monthly scale. Our multi‐scale temporal model allows for flexibility in the climate response through time at different temporal scales and predicts reasonable climate scenarios given tree ring width data. Copyright © 2015 John Wiley & Sons, Ltd.

On the dynamics of persistent states and their secular trends in the waveguides of the Southern Hemisphere troposphere

We identify the dynamical drivers of systematic changes in persistent quasi-stationary states (regimes) of the Southern Hemisphere troposphere and their secular trends. We apply a purely data-driven approach, whereby a multiscale approximation to nonstationary dynamical processes is achieved through optimal sequences of locally stationary fast vector autoregressive factor processes, to examine a high resolution atmospheric reanalysis over the period encompassing 1958–2013. This approach identifies regimes and their secular trends in terms of the predictability of the flow and is Granger causal. A comprehensive set of diagnostics on both isentropic and isobaric surfaces is employed to examine teleconnections over the full hemisphere and for a set of regional domains. Composite states for the hemisphere obtained from nonstationary nonparametric cluster analysis reveal patterns consistent with a circumglobal wave 3 (polar)–wave 5 (subtropical) pattern, while regional composites reveal the Pacific South American pattern and blocking modes. The respective roles of potential vorticity sources, stationary Rossby waves and baroclinic instability on the dynamics of these circulation modes are shown to be reflected by the seasonal variations of the waveguides, where Rossby wave sources and baroclinic disturbances are largely contained within the waveguides and with little direct evidence of sustained remote tropical influences on persistent synoptic features. Warm surface temperature anomalies are strongly connected with regions of upper level divergence and anticyclonic Rossby wave sources. The persistent states identified reveal significant variability on interannual to decadal time scales with large secular trends identified in all sectors apart from a region close to South America.

Skewness and kurtosis of multivariate Markov-switching processes

Exact formulae are provided for the calculation of multivariate skewness and kurtosis of Markov-switching Vector Auto-Regressive (MS VAR) processes as well as for the general class of MS state space (MS SS) models. The use of the higher-order moments in non-linear modeling is illustrated with two examples. A Matlab code that implements the results is available from the authors.

Identification of Lags in Nonlinear Autoregressive Time Series Using a Flexible Fuzzy Model

This work proposes a method to find the set of the most influential lags and the rule structure of a Takagi---Sugeno---Kang (TSK) fuzzy model for time series applications. The proposed method resembles the techniques that prioritize lags, evaluating the proximity of nearby samples in the input space using the closeness of the corresponding target values. Clusters of samples are generated, and the consistency of the mapping between the predicted variable and the set of candidate past values is evaluated. A TSK model is established, and possible redundancies in the rule base are avoided. The proposed method is evaluated using simulated and real data. Several simulation experiments were conducted for five synthetic nonlinear autoregressive processes, two nonlinear vector autoregressive processes and eight benchmark time series. The results show a competitive performance in the mean square error and a promising ability to find a proper set of lags for a given autoregressive process.