Stationary Gaussian Time Series Research Articles

Correlation Integral for Stationary Gaussian Time Series

Correlation Integral for Stationary Gaussian Time Series

On sparse high-dimensional graphical model learning for dependent time series

We consider the problem of inferring the conditional independence graph (CIG) of a sparse, high-dimensional stationary multivariate Gaussian time series. A sparse-group lasso-based frequency-domain formulation of the problem based on frequency-domain sufficient statistic for the observed time series is presented. We investigate an alternating direction method of multipliers (ADMM) approach for optimization of the sparse-group lasso penalized log-likelihood. We provide sufficient conditions for convergence in the Frobenius norm of the inverse PSD estimators to the true value, jointly across all frequencies, where the number of frequencies are allowed to increase with sample size. This result also yields a rate of convergence. We also empirically investigate selection of the tuning parameters based on the Bayesian information criterion, and illustrate our approach using numerical examples utilizing both synthetic and real data.

Spectral Independent Component Analysis with noise modeling for M/EEG source separation

BackgroundIndependent Component Analysis (ICA) is a widespread tool for exploration and denoising of electroencephalography (EEG) or magnetoencephalography (MEG) signals. In its most common formulation, ICA assumes that the signal matrix is a noiseless linear mixture of independent sources that are assumed non-Gaussian. A limitation is that it enforces to estimate as many sources as sensors or to rely on a detrimental PCA step. MethodsWe present the Spectral Matching ICA (SMICA) model. Signals are modelled as a linear mixing of independent sources corrupted by additive noise, where sources and the noise are stationary Gaussian time series. Thanks to the Gaussian assumption, the negative log-likelihood has a simple expression as a sum of ‘divergences’ between the empirical spectral covariance matrices of the signals and those predicted by the model. The model parameters can then be estimated by the expectation-maximization (EM) algorithm. ResultsOn phantom MEG datasets with low amplitude dipole sources (20 nAm), SMICA makes a median dipole localization error of 1.5 mm while competing methods make an error ≥7 mm. Experiments on EEG datasets show that SMICA identifies a source subspace which contains sources that have less pairwise mutual information, and are better explained by the projection of a single dipole on the scalp. With 10 sources, the number of strongly dipolar sources (dipolarity >90%) is more than 80% for SMICA while competing methods do not exceed 65%. Comparison with existing methodsWith the noisy model of SMICA, the number of sources to be recovered is controlled by choosing the size of the mixing matrix to be fitted rather than by a preprocessing step of dimension reduction which is required in traditional noise-free ICA methods. ConclusionsSMICA is a promising alternative to other noiseless ICA models based on non-Gaussian assumptions.

Edge Exclusion Tests for Graphical Model Selection: Complex Gaussian Vectors and Time Series

We consider the problem of inferring the conditional independence graph (CIG) of a stationary multivariate Gaussian time series. A $p$ -variate Gaussian time series graphical model associated with an undirected graph with $p$ vertices is defined as the family of time series that obey the conditional independence restrictions implied by the edge set of the graph. In some existing methods, partial coherence has been used as a test statistic for graphical model selection. To test inclusion/exclusion of a given edge in the graph, the test is applied at distinct frequencies, requiring multiple tests and leading to a loss in test power. The nonparametric method of Matsuda uses the Kullback-Leibler divergence measure to define a test statistic which does not need multiple testing to test between two competing models. In this paper we propose a generalized likelihood ratio test (GLRT) based edge exclusion test statistic that also does not need multiple testing. It is computationally significantly faster than the method of Matsuda, and simulations show that we achieve comparable power levels. Our proposed approach is based on a novel formulation of a GLRT based edge exclusion test for $p$ -variate complex Gaussian graphical models (CGGMs); this result is also of independent interest. The computational complexity of the proposed statistic is $\mathcal {O}(p^3)$ compared to $\mathcal {O}(p^5)$ for the existing result. We also apply our time series graphical model selection method to a foreign exchange data set consisting of monthly trends of foreign exchange rates of 10 countries.

Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

A new comprehensive approach to nonlinear time series analysis and modeling is developed in the present paper. We introduce novel data-specific mid-distribution-based Legendre Polynomial (LP)-like nonlinear transformations of the original time series {Y(t)} that enable us to adapt all the existing stationary linear Gaussian time series modeling strategies and make them applicable to non-Gaussian and nonlinear processes in a robust fashion. The emphasis of the present paper is on empirical time series modeling via the algorithm LPTime. We demonstrate the effectiveness of our theoretical framework using daily S&P 500 return data between 2 January 1963 and 31 December 2009. Our proposed LPTime algorithm systematically discovers all the ‘stylized facts’ of the financial time series automatically, all at once, which were previously noted by many researchers one at a time.

An Efficient Approach to Graphical Modeling of Time Series

A method for selecting a graphical model for p-vector-valued stationary Gaussian time series was recently proposed by Matsuda and uses the Kullback-Leibler divergence measure to define a test statistic. This statistic was used in a backward selection procedure, but the algorithm is prohibitively expensive for large p. A high degree of sparsity is not assumed. We show that reformulation in terms of a multiple hypothesis test reduces computation time by O(p <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and simulations support the assertion that power levels are attained at least as good as those achieved by Matsuda's much slower approach. Moreover, the new scheme is readily parallelizable for even greater speed gains.

Asymptotic independence of multiple Wiener–Itô integrals and the resulting limit laws

We characterize the asymptotic independence between blocks consisting of multiple Wiener–Itô integrals. As a consequence of this characterization, we derive the celebrated fourth moment theorem of Nualart and Peccati, its multidimensional extension and other related results on the multivariate convergence of multiple Wiener–Itô integrals, that involve Gaussian and non Gaussian limits. We give applications to the study of the asymptotic behavior of functions of short and long-range dependent stationary Gaussian time series and establish the asymptotic independence for discrete non-Gaussian chaoses.

On asymptotic properties of the plug-in cepstrum estimator for Gaussian time series

The paper considers probabilistic properties of the plug-in estimator of the cepstrum (Fourier transform of the logarithm of the spectral density) for stationary Gaussian time series. In the asymptotics of increasing observation time under some regularity condition we construct asymptotic expansions for the high-order central mixed moments of the log-periodogram and of the cepstrumestimator. These asymptotic expansions can be useful in signal processing for construction and performance analysis of statistical inference based on the plug-in estimator of the cepstrum.

ASYMPTOTIC THEORY FOR MAXIMUM LIKELIHOOD ESTIMATION OF THE MEMORY PARAMETER IN STATIONARY GAUSSIAN PROCESSES

Consistency, asymptotic normality, and efficiency of the maximum likelihood estimator for stationary Gaussian time series were shown to hold in the short memory case by Hannan (1973, Journal of Applied Probability 10, 130–145) and in the long memory case by Dahlhaus (1989, Annals of Statistics 34, 1045–1047). In this paper we extend these results to the entire stationarity region, including the case of antipersistence and noninvertibility.

Central Limit Theorems and Quadratic Variations in Terms of Spectral Density

We give a new proof and provide new bounds for the speed of convergence in the Central Limit Theorem of Breuer Major on stationary Gaussian time series, which generalizes to particular triangular arrays. Our assumptions are given in terms of the spectral density of the time series. We then consider generalized quadratic variations of Gaussian fields with stationary increments under the assumption that their spectral density is asymptotically self-similar and prove Central Limit Theorems in this context.

Fast and exact synthesis of stationary multivariate Gaussian time series using circulant embedding

A fast and exact procedure for the numerical synthesis of stationary multivariate Gaussian time series with a priori prescribed and well controlled auto- and cross-covariance functions is proposed. It is based on extending the circulant embedding technique to the multivariate case and can be viewed as a modification and variation around the Chan and Wood algorithm proposed earlier to solve the same problem. The procedure is shown to yield time series possessing exactly the desired covariance structure, when sufficient conditions are satisfied. Such conditions are discussed theoretically and examined on several examples of multivariate time series models. Issues related to prescribing a priori the spectral structure rather than the covariance one are also discussed. Matlab routines implementing this procedure are publicly available at http://www.hermir.org.

Intermittent Estimation for Gaussian Processes

Let {Xn}n=0 ? be a stationary real-valued Gaussian time series. We estimate the conditional expectation E(Xn+1|X0, ...,Xn) from a growing number of observations X0,..., Xn in a pointwise consistent way along a sequence of stopping times.

Statistical properties of detrended fluctuation analysis

The main goal of this work is to consider the detrended fluctuation analysis (DFA), proposed by Peng et al. [Mosaic organization of DNA nucleotides, Phys. Rev. E. 49(5) (1994), 1685–1689]. This is a well-known method for analysing the long-range dependence in non-stationary time series. Here we describe the DFA method and we prove its consistency and its exact distribution, based on the usual i.i.d. assumption, as an estimator for the fractional parameter d. In the literature it is well established that the nucleotide sequences present long-range dependence property. In this work, we analyse the long dependence property in view of the autoregressive moving average fractionally integrated ARFIMA(p, d, q) processes through the analysis of four nucleotide sequences. For estimating the fractional parameter d we consider the semiparametric regression method based on the periodogram function, in both classical and robust versions; the semiparametric R/S(n) method, proposed by Hurst [Long term storage in reservoirs, Trans. Am. Soc. Civil Eng. 116 (1986), 770–779] and the maximum likelihood method (see [R. Fox and M.S. Taqqu, Large-sample properties of parameter estimates for strongly dependent stationary Gaussian time series, Ann. Statist. 14 (1986), 517–532]), by considering the approximation suggested by Whittle [Hypothesis Testing in Time Series Analysis (1953), Hafner, New York].

A Bayesian approach to estimating the long memory parameter

We develop a Bayesian procedure for analyzing stationary long-range dependent processes. Specifically, we consider the fractional exponential model (FEXP) to estimate the memory parameter of a stationary long-memory Gaussian time series. In particular, we propose a hierarchical Bayesian model and make it fully adaptive by imposing a prior distribution on the model order. Further, we describe a reversible jump Markov chain Monte Carlo algorithm for variable dimension estimation and show that, in our context, the algorithm provides a reasonable method of model selection (within each repetition of the chain). Therefore, through an application of Bayesian model averaging, we incorporate all possible models from the FEXP class (up to a given finite order). As a result we reduce the underestimation of uncertainty at the model-selection stage as well as achieve better estimates of the long memory parameter. Additionally, we establish Bayesian consistency of the memory parameter under mild conditions on the data process. Finally, through simulation and the analysis of two data sets, we demonstrate the effectiveness of our approach.

Convergence rates of posterior distributions for noniid observations

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model.

Haar–Fisz Estimation of Evolutionary Wavelet Spectra

SummaryWe propose a new ‘Haar–Fisz’ technique for estimating the time-varying, piecewise constant local variance of a locally stationary Gaussian time series. We apply our technique to the estimation of the spectral structure in the locally stationary wavelet model. Our method combines Haar wavelets and the variance stabilizing Fisz transform. The resulting estimator is mean square consistent, rapidly computable and easy to implement, and performs well in practice. We also introduce the ‘Haar–Fisz transform’, a device for stabilizing the variance of scaled χ2-data and bringing their distribution close to Gaussianity.

Estimating seasonal long-memory processes: a Monte Carlo study

This paper discusses extensions of the popular methods proposed by Geweke and Porter-Hudak [Geweke, J. and Porter-Hudak, S., 1983, The estimation and application of long memory times series models. Journal of Time Series Analysis, 4(4), 221–238.] and Fox and Taqqu [Fox, R. and Taqqu, M.S., 1986, Large-sample properties of parameter estimates for strongly dependent stationary Gaussian time series. Annals of Statistics, 14, 517–532.] for estimating the long-memory parameter of autoregressive fractionally integrated moving average models to the estimation of long-range dependent models with seasonal components. The proposed estimates are obtained from a selection of harmonic frequencies chosen between the seasonal frequencies. The maximum likelihood method given in Beran [Beran, J., 1994, Statistic for Long-Memory Processes (New York: Chapman & Hall).] and the semi-parametric approaches introduced by Arteche and Robinson [Arteche, J. and Robinson, P.M., 2000, Semiparametric inference in seasonal and cyclical long memory processes. Journal of Time Series Analysis, 21(1), 1–25.] are also considered in the study. Our finite sample Monte Carlo investigations indicate that the proposed methods perform well and can be used as alternative estimating procedures when the data display both long-memory and cyclical behavior.

Higher-order improvements of the parametric bootstrap for long-memory Gaussian processes

Abstract This paper determines coverage probability errors of both delta method and parametric bootstrap confidence intervals (CIs) for the covariance parameters of stationary long-memory Gaussian time series. CIs for the long-memory parameter d 0 are included. The results establish that the bootstrap provides higher-order improvements over the delta method. Analogous results are given for tests. The CIs and tests are based on one or other of two approximate maximum likelihood estimators. The first estimator solves the first-order conditions with respect to the covariance parameters of a “plug-in” log-likelihood function that has the unknown mean replaced by the sample mean. The second estimator does likewise for a plug-in Whittle log-likelihood. The magnitudes of the coverage probability errors for one-sided bootstrap CIs for covariance parameters for long-memory time series are shown to be essentially the same as they are with iid data. This occurs even though the mean of the time series cannot be estimated at the usual n 1 / 2 rate.

Inverse Correlations for Multiple Time Series and Gaussian Random Fields and Measures of Their Linear Determinism

For a discrete-time vector linear stationary process, {X(t)}, admitting forward and backward autoregressive representations, the variance matrix of an optimal linear interpolator of X(t), based on a knowledge of {X(t-j), j≠0}, is known to be given by Ri(0)-1 where Ri(0) denotes the inverse variance of the process. Let A=Is-1Ri(0)ˉ1R(0)ˉ1, where R(0) denotes the variance matrix of {X(t)} and Is an sXs, identity matrix. A measure of linear interpolability of the process, called an index of linear determinism, may be constructed from the determinant Det[Is - A], of Is - A = Ri(0)-1 R(0)-1. An alternative measure is constructed by relating tr[Ri(0)-1] the trace of Ri(0)-1, to tr[R(0)]. The relationship between the matrix A and the corresponding matrix, P, obtained by considering only an optimal one-step linear predictor of X(t) from a knowledge of its infinite past, {X(t-j),j>0}, is also discussed. The possible role the inverse correlation function may have for model specification of a vector ARMA model is explored. Close parallels between the problem of interpolation for a stationary univariate two-dimensional Gaussian random field and time series are examined and an index of linear determinism for the latter class of processes is also defined. An application of this index for model specification and diagnostic testing of a Gaussian Markov Random Field is investigated together with the question of its estimation from observed data. Results are illustrated by a simulation study.

Wavelet-Based Bootstrap for Time Series Analysis

ABSTRACT Using wavelet domain analysis and modeling of stochastic processes, we develop a unified bootstrap scheme that can be applied to both short-range dependent and long-range dependent stationary Gaussian time series. Our idea was motivated by the fact that the discrete wavelet transform is capable of converting long-range dependence in the time domain into short-range dependence in the wavelet domain. Hence we can use a simple Markov model to model the short-range dependence in the wavelet domain. The Markov model is used to generate a new version (the bootstrapped version) of the wavelet representation of the time series. The bootstrapped series is obtained by performing the inverse wavelet transform on the new wavelet representation of the original series. We compare our wavelet-based bootstrap with the moving block bootstrap for estimating the standard errors of the unit lag sample autocorrelation and the sample standard deviation. Our results show that the wavelet-based bootstrap can achieve performance better than the moving block bootstrap for both short-range dependent data and long-range dependent data.