Multivariate Autoregressive Model Research Articles

Graph-based multivariate conditional autoregressive models

ABSTRACTThe conditional autoregressive model is a routinely used statistical model for areal data that arise from, for instances, epidemiological, socio-economic or ecological studies. Various multivariate conditional autoregressive models have also been extensively studied in the literature and it has been shown that extending from the univariate case to the multivariate case is not trivial. The difficulties lie in many aspects, including validity, interpretability, flexibility and computational feasibility of the model. In this paper, we approach the multivariate modelling from an element-based perspective instead of the traditional vector-based perspective. We focus on the joint adjacency structure of elements and discuss graphical structures for both the spatial and non-spatial domains. We assume that the graph for the spatial domain is generally known and fixed while the graph for the non-spatial domain can be unknown and random. We propose a very general specification for the multivariate conditional modelling and then focus on three special cases, which are linked to well-known models in the literature. Bayesian inference for parameter learning and graph learning is provided for the focused cases, and finally, an example with public health data is illustrated.

Recurrent neural networks for reconstructing complex directed brain connectivity.

While Granger Causality(GC)-based approaches have been widely employed in a vast number of problems in network science, the vast majority of GC applications are based on linear multivariate autoregressive (MVAR) models. However, it is well known that real-life system (and biological networks in particular) exhibit notable nonlinear behavior, hence undermining that validity of MVAR-based approaches to estimating GC (MVAR-GC). In this paper, we define a novel approach to estimating nonlinear, directed within-network interactions based on a specific class of recurrent neural networks (RNN) termed echo-state networks (ESN). We reformulate the classical GC framework in terms of ESN-based models for multivariate signals generated by arbitrarily complex networks, and characterize the ability of our ESN-based Granger Causality (ES-GC) to capture nonlinear causal relations by simulating multivariate coupling in a network of nonlinearly interacting, noisy Duffing oscillators operating in a chaotic regime. Synthetic validation shows a net advantage of ES-GC over all other estimators in detecting nonlinear, causal links. We then explore the structure of EC-GC networks in the human brain in functional MRI data from 1003 healthy subjects scanned at rest at 3T, discovering previously unknown between-network interactions. In summary, ES-GC performs significantly better than commonly used and recently developed GC detection tools, making it a superior tool for the analysis of e.g. multivariate biological networks.

Single-trial Connectivity Estimation through the Least Absolute Shrinkage and Selection Operator.

Methods based on the use of multivariate autoregressive models (MVAR) have proved to be an accurate tool for the estimation of functional links between the activity originated in different brain regions. A well-established method for the parameters estimation is the Ordinary Least Square (OLS) approach, followed by an assessment procedure that can be performed by means of Asymptotic Statistic (AS). However, the performances of both procedures are strongly influenced by the number of data samples available, thus limiting the conditions in which brain connectivity can be estimated. The aim of this paper is to introduce and test a regression method based on Least Absolute Shrinkage and Selection Operator (LASSO) to broaden the estimation of brain connectivity to those conditions in which current methods fail due to the limited data points available. We tested the performances of the LASSO regression in a simulation study under different levels of data points available, in comparison with a classical approach based on OLS and AS. Then, the two methods were applied to real electroencephalographic (EEG) signals, recorded during a motor imagery task. The simulation study and the application to real EEG data both indicated that LASSO regression provides better performances than the currently used methodologies for the estimation of brain connectivity when few data points are available. This work paves the way to the estimation and assessment of connectivity patterns with limited data amount and in on-line settings.

A Parsimonious Granger Causality Formulation for Capturing Arbitrarily Long Multivariate Associations.

High-frequency neuroelectric signals like electroencephalography (EEG) or magnetoencephalography (MEG) provide a unique opportunity to infer causal relationships between local activity of brain areas. While causal inference is commonly performed through classical Granger causality (GC) based on multivariate autoregressive models, this method may encounter important limitations (e.g., data paucity) in the case of high dimensional data from densely connected systems like the brain. Additionally, physiological signals often present long-range dependencies which commonly require high autoregressive model orders/number of parameters. We present a generalization of autoregressive models for GC estimation based on Wiener–Volterra decompositions with Laguerre polynomials as basis functions. In this basis, the introduction of only one additional global parameter allows to capture arbitrary long dependencies without increasing model order, hence retaining model simplicity, linearity and ease of parameters estimation. We validate our method in synthetic data generated from families of complex, densely connected networks and demonstrate superior performance as compared to classical GC. Additionally, we apply our framework to studying the directed human brain connectome through MEG data from 89 subjects drawn from the Human Connectome Project (HCP) database, showing that it is able to reproduce current knowledge as well as to uncover previously unknown directed influences between cortical and limbic brain regions.

Detecting connectivity in EEG: A comparative study of data-driven effective connectivity measures

In this paper, we perform the first comparison of a large variety of effective connectivity measures in detecting causal effects among observed interacting systems based on their statistical significance. Well-known measures estimating direction and strength of interdependence between time series are compared: information theoretic measures, model-based multivariate measures in the time and frequency domains, and phase-based measures. The performance of measures is tested on simulated data from three systems: three coupled Hénon maps; a multivariate autoregressive (MVAR) model with and without EEG as an exogenous input; and simulated EEG. No measure was consistently superior. Measures that model the data as MVAR perform well when the data are drawn from that model. Frequency domain measures perform well when the data have a clearly defined band of interest. When neither of these is true, information theoretic measures perform well. Overall, the measure with the best performance in a variety of situations and with a low computational cost is conditional Granger causality. Partial Granger causality and multivariate Granger causality are also good measures, but their computational cost rises rapidly with the number of channels. Copula Granger causality can also be used reliably, but its computational cost rises rapidly with the number of data.

Causal Coupling Between Electrophysiological Signals, Cerebral Hemodynamics and Systemic Blood Supply Oscillations in Mayer Wave Frequency Range.

The aim of the study was to assess causal coupling between neuronal activity, microvascular hemodynamics and blood supply oscillations in the Mayer wave frequency range. An electroencephalogram, cerebral blood oxygenation changes, an electrocardiogram and blood pressure were recorded during rest and during a movement task. Causal coupling between them was evaluated using directed transfer function, a measure based on the Granger causality principle. The multivariate autoregressive model was fitted to all the signals simultaneously, which made it possible to construct a complete scheme of interactions between the considered signals. The obtained pattern of interactions in the resting state estimated in the 0.05-0.15 Hz band revealed a predominant influence of blood pressure oscillations on all the other variables. Reciprocal connections between blood pressure and heart rate variability time series indicated the presence of feedback loops between these signals. During movement, the pattern of connections did not change dramatically. The number of connections decreased, but the couplings between blood pressure and heart rate variability signal were not significantly changed, and the strong influence of the decreased blood hemoglobin concentration on the oxygenated hemoglobin concentration persisted. For the first time our results provided a comprehensive scheme of interactions between electrical and hemodynamic brain signals, heart rate and blood pressure oscillations. Persistent reciprocal connections between blood pressure and heart rate variability time series suggest possible feedforward and feedback coupling of cardiovascular variables which may lead to the observed oscillations in Mayer wave range.

Comparison and assessment of large-scale surface temperature in climate model simulations

Abstract. We present a data-driven approach to assess and compare the behavior of large-scale spatial averages of surface temperature in climate model simulations and in observational products. We rely on univariate and multivariate dynamic linear model (DLM) techniques to estimate both long-term and seasonal changes in temperature. The residuals from the DLM analyses capture the internal variability of the climate system and exhibit complex temporal autocorrelation structure. To characterize this internal variability, we explore the structure of these residuals using univariate and multivariate autoregressive (AR) models. As a proof of concept that can easily be extended to other climate models, we apply our approach to one particular climate model (MIROC5). Our results illustrate model versus data differences in both long-term and seasonal changes in temperature. Despite differences in the underlying factors contributing to variability, the different types of simulation yield very similar spectral estimates of internal temperature variability. In general, we find that there is no evidence that the MIROC5 model systematically underestimates the amplitude of observed surface temperature variability on multi-decadal timescales – a finding that has considerable relevance regarding efforts to identify anthropogenic “fingerprints” in observational surface temperature data. Our methodology and results present a novel approach to obtaining data-driven estimates of climate variability for purposes of model evaluation.

A self-organized recurrent neural network for estimating the effective connectivity and its application to EEG data

ObjectiveEffective connectivity is an important notion in neuroscience research, useful for detecting the interactions between regions of the brain. New methodSince we are dealing with a dynamic system, it seems that using a dynamic tool could effectively achieve better results. In this paper, a novel approach, called “Recurrent Neural Network - Neuron Growth Using Error Whiteness - Granger Causality” (RNN-NGUEW-GC) is proposed to estimate the effective connectivity. An RNN is used for predicting and modeling time series and multivariate signals. NGUEW is used to determine the optimum time lag with the help of an error whiteness criterion. When this criterion is not satisfied, the number of neurons in the network input is increased, producing an increase in the time lag. Accordingly, the network achieves a self-organized structure. Finally, causal effects are determined for linear and nonlinear models using the concept of Granger causality. Also, an indicator of the ‘‘intensity of causality’’ is defined to approximate the strength of the linear interactions based on the structure of the network and the weights of the connections. ConclusionsRNN-NGUEW-GC had a major outcome in terms of both method accuracy on simulation data and prediction of epileptic seizures on the EEG dataset. The main advantages of this method in comparison with other methods of determining the effective connectivity are: 1) there is no need for physiological information; 2) it yields a self-organized network structure. In addition, the calculation of the appropriate time lag using NGUEW is another superiority of this method in comparison with multivariate auto-regressive models.

On the Constancy of Hedonic Wine Price Coefficients over Time

AbstractThis article tests the stability of the main hedonic wine price coefficients over time. We draw on an extensive literature review to identify the most frequently used methodology and define a standard hedonic model. We estimate this model on monthly subsamples of a worldwide auction database of the most commonly exchanged fine wines. This provides, for each attribute, a monthly time series of hedonic coefficients time series data from 2003 to 2014. Using a multivariate autoregressive model, we then study the stability of these coefficients over time and test the existence of structural or cyclical changes related to fluctuations in general price levels. We find that most hedonic coefficients are variable and either exhibit structural or cyclical variations over time. These findings shed doubt on the relevance of both short- and long-run hedonic estimations. (JEL Classifications: C13, C22, D44, G11)

Spatiotemporal Heterogeneity in the Distribution of Chikungunya and Zika Virus Case Incidences during their 2014 to 2016 Epidemics in Barranquilla, Colombia.

Chikungunya virus (CHIKV) and Zika virus (ZIKV) have recently emerged as globally important infections. This study aimed to explore the spatiotemporal heterogeneity in the occurrence of CHIKV and ZIKV outbreaks throughout the major international seaport city of Barranquilla, Colombia in 2014 and 2016 and the potential for clustering. Incidence data were fitted using multiple Bayesian Poisson models based on multiple explanatory variables as potential risk factors identified from other studies and options for random effects. A best fit model was used to analyse their case incidence risks and identify any risk factors during their epidemics. Neighbourhoods in the northern region were hotspots for both CHIKV and ZIKV outbreaks. Additional hotspots occurred in the southwestern and some eastern/southeastern areas during their outbreaks containing part of, or immediately adjacent to, the major circular city road with its import/export cargo warehouses and harbour area. Multivariate conditional autoregressive models strongly identified higher socioeconomic strata and living in a neighbourhood near a major road as risk factors for ZIKV case incidences. These findings will help to appropriately focus vector control efforts but also challenge the belief that these infections are driven by social vulnerability and merit further study both in Barranquilla and throughout the world’s tropical and subtropical regions.

Multivariate Quantile Impulse Response Functions

A reduced form multivariate quantile autoregressive model is developed to study heterogeneity in the effects of macroeconomic shocks. This framework is used for forecasting and for constructing quantile impulse response functions that explore dynamic heterogeneity in the response of endogenous variables to different shocks. The methodology allows evaluating different quantile paths, defined as the dynamic effects for a fix collection of quantile indexes. The model is applied to study monetary shocks in a three‐variable macroeconomic model (output gap, inflation, Fed Funds rate) for the USA for the period 1980q1–2010q1.

A Threshold Multivariate Model to Explain Fiscal Multipliers with Government Debt

This paper shows fiscal multipliers, considering levels of public debt with multivariate threshold models. Non-linear behavior in sovereign debt-to-GDP ratio time series determine the relationship between output and government expenditure. The debt-to-GDP ratio has been selected optimally as an endogenous threshold variable to evaluate non-linearities; it has been useful for identifying estimators in a multivariate threshold autoregressive model; and it has been an important tool to observe how the multiplier changes during good times and bad. Expansionary fiscal policies seem to be counterproductive in this framework. This result highlights the link between real and financial variables.

A Novel Framework for Estimating Time-Varying Multivariate Autoregressive Models and Application to Cardiovascular Responses to Acute Exercise.

We present a novel modeling framework for identifying time-varying (TV) couplings between time-series of biomedical relevance. The proposed methodology is based on multivariate autoregressive (MVAR) models, which have been extensively used to study couplings between biosignals. Contrary to the standard estimation methods that assume time-invariant relationships, we propose a modified recursive Kalman filter (KF) to track changes in the model parameters. We perform model order selection and hyperparameter optimization simultaneously using Genetic Algorithms, greatly improving accuracy and computation time. In addition, we address the effect of residual heteroscedasticity, possibly associated with event-related changes or phase transitions during a given experimental protocol, on the TV-MVAR coupling measures by using Generalized Autoregressive Conditional Heteroskedasticity (GARCH) models to fit the TV-MVAR residuals. Using simulated data, we show that the proposed framework yields more accurate parameter estimates compared to the conventional KF, particularly when the true system parameters exhibit different rate of variations over time. Furthermore, by accounting for heteroskedasticity, we obtain more accurate estimates of the strength and directionality of the underlying couplings. We also use our approach to investigate TV hemodynamic interactions during exercise in young and old healthy adults, as well as individuals with chronic stroke. We extract TV coupling patterns that reflect well known exercise-induced effects on the underlying regulatory mechanisms with excellent time resolution. The proposed modeling framework can be used to efficiently quantify TV couplings between biosignals. It is fully automated and does not require prior knowledge of the system TV characteristics.

Exploring directed functional connectivity based on electroencephalography source signals using a global cortex factor-based multivariate autoregressive model

BackgroundPartial directed coherence (PDC) computed from multivariate autoregressive (MVAR) model coefficient is increasingly being used to study directed functional connectivity between brain regions in the frequency domain based on electroencephalography (EEG) source signals. However, directly fitting MVAR model to the high-dimensional source signals is difficult. Besides, although PDC measurement often shows good results for simulated data, it is not clear to what extent the results for real data are physiologically plausible. New methodWe propose a new method termed global cortex factor-based MVAR (GCF-MVAR) to study directed functional connectivity based on EEG source signals. It avoids directly fitting MVAR model to high-dimensional EEG sources, instead using low-dimensional global cortex factor signals derived from the source signals by principle component analysis (PCA). To validate its physiological efficacy, we weight the PDC with source spectral power (SP) which reflects the true frequency activity in a source region. Comparison with existing method(s)The performance of GCF-MVAR is compared with FMVAR, ROI-MVAR, and MVAR by applying to both simulated and resting-state EEG data. ResultsThe simulation results show that GCF-MVAR has the lowest estimation error. By using the source SP to weight the PDC, GCF-MVAR improves the physiological interpretation of the source connectivity for both simulated and resting-state EEG data. ConclusionsThe new method is proved to outperform than the state-of-the-art methods, and can be feasible not only for resting state studies, but also task-related connectivity and neurological disorder analysis.

Estimation of Sparse Directional Connectivity With Expectation Maximization

Motivated by the many applications associated with estimation of sparse multivariate models, the estimation of sparse directional connectivity between the imperfectly measured nodes of a network is studied. Node dynamics and interactions are assumed to follow a multivariate autoregressive model driven by noise, and the observations are a noisy linear combination of the underlying node activities. The corresponding maximum a posteriori (MAP) problem is derived to estimate system parameters. Due to the intractability of the MAP problem, the expectation maximization (EM) framework is used to iteratively implement the MAP estimation. To impose sparsity, the EM algorithm is augmented with an $\ell _1$ regularization of the connectivity matrix. Multiple techniques have been used to lower the computational complexity. Importantly, an efficient coordinate descent algorithm utilizing a closed-form solution is designed to solve the $\ell _1$ -regularized sub-problem of the EM. Cholesky factors of the unknown covariance matrices are used directly in the optimization process in order to impose positive definiteness and guarantee the functionality of the $\ell _1$ optimization. The algorithm is first applied to synthetic data to evaluate the estimation accuracy. A comparison with the previous work over an extensive set of configurations shows that our method is superior under moderate to high sparsity. Finally, the algorithm is applied to real temperature data obtained from 98 stations across the U.S. mainland to identify the predictive interactions between the time series.

Modeling and Forecasting the Multivariate Realized Volatility of Financial Markets with Time-Varying Sparsity

ABSTRACTWe develop a Multivariate Heterogeneous Autoregressive (MHAR) model with time-varying sparsity (TVS), or the TVS-MHAR-X model, to model and forecast the realized covariance matrices by employing the data from China’s financial markets. We employ the matrix decomposition method to ensure the positivity of the forecasted covariance matrix and incorporate a set of predictors including the lagged daily, weekly and monthly volatilities, the leverage variables, and the jump variables. The proposed model allows the sparsity of coefficients to change over time based on the importance of predictors. We compare the forecast performances of the proposed models with the competing models based on the statistical evaluation and the economic evaluation. The results show that the proposed MHAR-TVS-X model outperforms the competing models for the short-term forecasts in terms of statistical evaluation. The results also suggest that the MHAR-TVS-X model significantly improves the efficient frontier and economic values for the short-term and long-term forecasts in terms of economic evaluation.

Scenario generation and probabilistic forecasting analysis of spatio-temporal wind speed series with multivariate autoregressive volatility models

This paper evaluates the application of a family of VAR-mGARCH (Vector AutoRegressive with multivariate AutoRegressive Conditional Heteroskedasticity) volatility models to the problem of modeling (i.e. generating correct in-sample scenarios) and forecasting in a probabilistic way three univariate but mutually dependent wind speed hourly series. The evaluation starts from the consideration that optimal modeling and optimal forecasting are not necessarily attained by the same models, and that they are assessed in different ways. The proposed VAR-mGARCH family, originated in Finance, consists of the VAR-BEKK (VAR - Baba, Engle, Kraft and Kroner) model and the VAR-DCC (VAR - Dynamic Conditional Correlation) model, the latter seen as the simplified and more scalable version of the former. These models were never used in wind speed studies before. Both models will be used with Gaussian innovations, and the VAR-DCC model will be also used with Student’s t innovations. These models, which are able to dynamically couple volatilities, are compared to two benchmark model sets, the first set consisting of three independent univariate AR-GARCH (i.e. non-vector) models, the second set consisting of an individual VAR model without the GARCH sector. These benchmark models cannot dynamically couple volatilities. Both benchmark model sets will be used with Gaussian innovations. The time series used for the evaluation are taken from three North-American wind metering sites located at few tens of kilometers from each other. In order to highlight the usefulness of choosing autoregressive coupled volatility schemes for modeling and forecasting spatially and temporally varying wind speed data, a controlled variance electricity generation portfolio example of Markowitz type, typical of Energy Finance, is also discussed at some length. In all, it is concluded that the VAR-DCC model with Student’s t innovations is the most balanced choice for this data set when the three goals of modeling, forecasting and scalability in the number of considered sites are taken into account at once.

Neural correlates of online cooperation during joint force production

During joint action, two or more persons depend on each other to accomplish a goal. This mutual recursion, or circular dependency, is one of the characteristics of cooperation. To evaluate the neural substrates of cooperation, we conducted a hyperscanning functional MRI study in which 19 dyads performed a joint force-production task. The goal of the task was to match their average grip forces to the target value (20% of their maximum grip forces) through visual feedback over a 30-s period; the task required taking into account other-produced force to regulate the self-generated one in real time, which represented cooperation. Time-series data of the dyad's exerted grip forces were recorded, and the noise contribution ratio (NCR), a measure of influence from the partner, was computed using a multivariate autoregressive model to identify the degree to which each participant's grip force was explained by that of their partner's, i.e., the degree of cooperation. Compared with the single force-production task, the joint task enhanced the NCR and activated the mentalizing system, including the medial prefrontal cortex, precuneus, and bilateral posterior subdivision of the temporoparietal junction (TPJ). In addition, specific activation of the anterior subdivision of the right TPJ significantly and positively correlated with the NCR across participants during the joint task. The effective connectivity of the anterior to posterior TPJ was upregulated when participants coordinated their grip forces. Finally, the joint task enhanced cross-brain functional connectivity of the right anterior TPJ, indicating shared attention toward the temporal patterns of the motor output of the partner. Since the posterior TPJ is part of the mentalizing system for tracking the intention of perceived agents, our findings indicate that cooperation, i.e., the degree of adjustment of individual motor output depending on that of the partner, is mediated by the interconnected subdivisions of the right TPJ.

Using multivariate state-space models to examine commercial stocks of redfish (Sebastes spp.) on the Flemish Cap

There are three different species of redfish (Sebastes spp.) in the waters of the Flemish Cap (Division 3M, NAFO Regulatory Area): S. fasciatus, S. mentella, and S. norvegicus. Historically, S. fasciatus and S. mentella have been managed together as a single stock because of similar biology and difficulty in species identification. Here we use multivariate autoregressive state-space models to examine the abundance trajectories of the three species and to determine whether they can be treated as a single stock for management purposes or whether they should be treaty separately. We also included covariates to evaluate relationships with climate, commercial catch, and the abundance of predators and (or) competitors and prey. We did two separate analyses: (i) a single-period analysis over the full time series and (ii) a blocked, two-period analysis over different regulatory periods. In both analyses, the best-fit model included separate trajectories for each species at each depth but one overall stock growth rate; both also included commercial catches as a covariate. These analyses suggest that a single assessment for the Sebastes complex is acceptable.

Estimating feedforward and feedback effective connections from fMRI time series: Assessments of statistical methods.

We test the adequacies of several proposed and two new statistical methods for recovering the causal structure of systems with feedback from synthetic BOLD time series. We compare an adaptation of the first correct method for recovering cyclic linear systems; Granger causal regression; a multivariate autoregressive model with a permutation test; the Group Iterative Multiple Model Estimation (GIMME) algorithm; the Ramsey et al. non-Gaussian methods; two non-Gaussian methods by Hyvärinen and Smith; a method due to Patel et al.; and the GlobalMIT algorithm. We introduce and also compare two new methods, Fast Adjacency Skewness (FASK) and Two-Step, both of which exploit non-Gaussian features of the BOLD signal. We give theoretical justifications for the latter two algorithms. Our test models include feedback structures with and without direct feedback (2-cycles), excitatory and inhibitory feedback, models using experimentally determined structural connectivities of macaques, and empirical human resting-state and task data. We find that averaged over all of our simulations, including those with 2-cycles, several of these methods have a better than 80% orientation precision (i.e., the probability of a directed edge is in the true structure given that a procedure estimates it to be so) and the two new methods also have better than 80% recall (probability of recovering an orientation in the true structure).