Yule-Walker Estimators Research Articles

Closed-form unbiased frequency estimation of a noisy sinusoid using notch filters

In this note, the problem of the frequency estimation of a sinusoid embedded in white noise is considered. The approach used herein is the minimization of the sample variance of the output of constrained notch filters fed by the noisy sinusoid. In particular, this note focuses on closed-form expressions of the frequency estimate, which can be obtained using notch filters having an all-zeros finite-impulse response (FIR) structure. The results presented in this note are as follows: 1) it is shown that the FIR notch filters obtained from standard second-order infinite-impulse response (IIR) filters are inadequate; 2) a new second-order IIR notch filter is proposed, which provides an unbiased estimate of the frequency; 3) the FIR filter obtained from the new IIR filter provides a closed-form unbiased frequency estimate; and 4) the closed-form frequency estimate obtained using the new FIR notch filter asymptotically converges toward the Pisarenko harmonic decomposition estimator and the Yule-Walker estimator.

On finite sample statistics for Yule-Walker estimates

The expectation of the square of the reflection coefficient in small samples of white noise is derived. This variance of the reflection coefficient in white noise is a key factor in the statistics of the estimation of autoregressive (AR) models in small samples. Approximations of this expectation are constructed that are more accurate than the known first-order Taylor approximation. These better approximations are needed, because in some applications (radar applications, for example) the number of observations is small (say 63 observations) and asymptotic descriptions do not cover the estimates.

Implementing Modifed Burg Algorithms in Multivariate Subset Autoregressive Modeling

The large number of parameters in subset vector autoregressive models often leads one to procure fast, simple, and efficient alternatives or precursors to maximum likelihood estimation. We present the solution of the multivariate subset Yule-Walker equations as one such alternative. In recent work, Brockwell, Dahlhaus, and Trindade (2002), show that the Yule-Walker estimators can actually be obtained as a special case of a general recursive Burg-type algorithm. We illustrate the structure of this Algorithm, and discuss its implementation in a high-level programming language. Applications of the Algorithm in univariate and bivariate modeling are showcased in examples. Univariate and bivariate versions of the Algorithm written in Fortran 90 are included in the appendix, and their use illustrated.

Asymptotic laws of successive least squares estimates for seasonal arima models and application

Abstract. In view of detecting the stochastic non‐stationarity in time series, successive Yule–Walker estimates are considered for general seasonal ARIMA models and their asymptotic laws are obtained. This extends results known on least squares estimates for stable–unstable ARMA. Furthermore, these asymptotic laws are then compared with analogous results obtained under some additive seasonal model that corresponds to the case of deterministic seasonal behaviour. These results, combined with a simulation study, reveal that successive autoregressions provide a very useful tool both for identifying seasonal ARIMA processes and for distinguishing between stochastic and deterministic seasonal behaviours.

Estimation de Yule–Walker d'un CAR( p) observé à temps discret

Let (X(lδ), l=0,n) be a discrete observation at mesh δ>0 of X, a CAR( p). Classical Yule–Walker estimation are biased and must be corrected. Resultant estimators converge if T= nδ→+∞, are asymptotically normal with rate T , and efficient. The diffusion coefficient is also estimated, with rate Tδ −1 .

Efficient use of higher‐lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time‐series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher‐order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first‐order autoregressive process with a Markov‐switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.

EDGEWORTH APPROXIMATION IN THE AR(1) PROCESS WITH SOME POSSIBLY NONZERO INITIAL VALUE

where c1, c2 ≥ 0 (see Ochi (1983)). The estimator θ1,0 is the least-squares estimator of θ which minimizes ∑T t=2(Xt − θXt−1), and for the model (1.1) with initial value X0 = 0, it is also the maximum likelihood estimator of θ. Furthermore, the estimator θ1,1 is the Yule-Walker estimator, and the estimator θ1/2,1/2 is the Burg estimator (see also Daniels (1956)). The random variable θc1,c2 is a ratio of quadratic forms of correlated normal random variables. The methods for calculating the probability distribution of such ratios are given in Mathai and Provost (1992; Section 4.5). The formulae they present are complicated, and one resorts instead to numerical methods such as Imhof (1961) and Helstrom (1996). Imhof (1961) gave a numerical method for inverting the characteristic function of a linear combination of several noncentral chi-squares random variables, and Helstrom (1996) used the saddlepoint integration for the numerical inversion. The saddlepoint approximation method in statistics was originally introduced by Daniels (1954, 1956). Following Daniels (1956), Phillips (1978) and Kakizawa (1996) derived the saddlepoint density for

THE ASYMPTOTIC JOINT DISTRIBUTION OF THE YULE-WALKER ESTIMATORS OF A CAUSAL MULTIDIMENSIONAL AR PROCESS

The Yule-Walker estimators of the AR coefficients of a causal multidimensional AR model are obtained by replacing the autocovariances with their estimators in the Yule-Walker equations. It is shown that only unbiased-type estimators of the autocovariances yield consistency of the Yule-Walker estimators. Also, the asymptotic joint distribution of the Yule-Walker estimators is presented.

Pitfalls of Fitting Autoregressive Models for Heavy-tailed Time Series

We consider the analysis of time series data which require models with a heavy-tailed marginal distribution. A natural model to attempt to fit to time series data is an autoregression of order p, where p itself is often determined from the data. Several methods of parameter estimation for heavy tailed autoregressions have been considered, including Yule–Walker estimation, linear programming estimators, and periodogram based estimators. We investigate the statistical pitfalls of the first two methods when the models are mis-specified—either completely or due to the presence of outliers. We illustrate the results of our considerations on both simulated and real data sets. A warning is sounded against the assumption that autoregressions will be an applicable class of models for fitting heavy tailed data.

On the Optimal Segment Length for Parameter Estimates for Locally Stationary Time Series

We discuss the behaviour of parameter estimates when stationary time series models are fitted locally to non‐stationary processes which have an evolutionary spectral representation. A particular example is the estimation for an autoregressive process with time‐varying coefficients by local Yule–Walker estimates. The bias and the mean squared error for the parameter estimates are calculated and the optimal length of the data segment is determined.

On the Statistical Inference of a Machine-generated Autoregressive AR(1) Model

Summary We have obtained the asymptotic bias and the limiting distribution for the Yule–Walker estimator of the autoregressive parameter under a considerably weaker assumption than that of independence in the noise sequence. Among other things, these suggest robustness of the classical results and throw some light on the use of simulations based on pseudorandom numbers in verifying these results.

Estimating the parameters of the generalized poisson AR(1) process

The main objective of this paper is to compare two methods of estimating the parameters of the Generalized Poisson Autoregressive Process of order one. The small sample performance of the methods of Yule-Walker and Gaussian Estimation is studied. A simulation study is presented to compare the two methods. An application of the model to data representing the numbers of failures of a computer system is given.

Calculating the Distribution of the Serial Correlation Estimator by Saddlepoint Integration

The efficient method of numerical saddlepoint integration is described and applied to calculating the probability distribution of the maximum likelihood and Yule-Walker estimators of the correlation coefficient a of a first-order autoregressive normal time series with initial value either zero or nonzero when a finite number n of data are at hand. Stationary time series of the same type are also treated. Significance points are computed in a number of examples to show how, as n increases, the finite-sample distributions approach the asymptotic distributions that have appeared in the literature.

Limit distributions for linear programming time series estimators

We consider stationary autoregressive processes of order p which have positive innovations. We propose consistent parameter estimators based on linear programming. Under conditions, including regular variation of either the left or right tail of the innovations distribution, we prove that the estimators have a limit distribution. The rate of convergence of our estimator is favorable compared with the Yule—Walker estimator under comparable circumstances.

On the covariance matrix estimators of the white noise process of a vector autoregressive model

The penalty function identification methods such as the AIC, the CAT, the BIC and Hannan and Quinn's criterion have been used to determine the order of a vector autoregressive model. To use the methods, it is necessary to calculate the maximum likelihood estimates of the covariance matrices of the white noise processes of several vector autoregressive models. Since the maximum likelihood estimation needs a lot of calculation, the Yule-Walker estimators of the white noise covariance matrices are used instead. In this paper the biases of the maximum likelihood estimator and the Yule-Walker estimator of the white noise covariance matrix are derived up to O(1/T 2), where T is the number of observations. Also, new estimators based on either the maximum likelihood estimator or the Yule-Walker estimator, which are less biased than the original ones, are proposed to identify the vector autoregressive model more adequately through the penalty function identification methods.

High-order Yule-Walker estimation of the parameters of exponentially damped cisoids in noise

An approach for the estimation of the frequencies and damping factors of exponentially damped cisoids (complex sinusoids) is presented. The method may be seen as an extension of the method of backward linear prediction and singular value decomposition of Kumaresan and Tufts to the second-order statistics domain. The proposed estimator is interpreted as a high-order Yule-Walker (HOYW) method using a data based covariance matrix. The HOYW method is analysed at high SNR where closed-form expressions for the accuracy of the estimates are derived. By Monte Carlo simulations the HOYW method is applied to data consisting of one and two damped cisoids in additive white noise. The simulation results are compared with the results using the Kumaresan and Tufts method, with the Cramér-Rao bound, and with the derived theoretical results. The method is not statistically efficient, but the comparison shows that the HOYW method outperforms the method of Kumaresan and Tufts in terms of accuracy versus algorithmic complexity and in terms of precision in the cases considered. Due to the above properties the method is suitable to provide fast initial estimates for nonlinear search methods.

The Bias of Estimators of Causal Spatial Autoregressive Processes

We study the asymptotic distribution of Yule-Walker and least squares estimators for twodimensional causal autoregressive processes observed on a rectangular part of a lattice. An explicit expression for the asymptotic bias of Yule-Walker estimators is obtained. It is shown that this bias disappears if we use the so-called unbiased sample autocovariance function or least squares estimators.

The bias of estimators of causal spatial autoregressive processes

SUMMARY We study the asymptotic distribution of Yule-Walker and least squares estimators for twodimensional causal autoregressive processes observed on a rectangular part of a lattice. An explicit expression for the asymptotic bias of Yule-Walker estimators is obtained. It is shown that this bias disappears if we use the so-called unbiased sample autocovariance function or least squares estimators.

A note on properties of spatial yule-walker estimators

For two-dimensional spatial autoregressive (AR) models, asymptotic properties of the spatial Yule-Walker (YW) estimators (Tjostheim, 1978) are studied. These estimators although consistent, are shown to be asymptotically biased. Estimators from the first-order spatial bilateral AR model are looked at in more detail and the spatial YW estimators for this model are compared with the exact maximum likelihood estimators. Small sample properties of both estimators are also discussed briefly and some simulation results are presented.

ON THE ASYMPTOTIC DISTRIBUTION OF THE GENERALIZED PARTIAL AUTOCORRELATION FUNCTION IN AUTOREGRESSIVE MOVING‐AVERAGE PROCESSES

Abstract. It has been conjectured and illustrated that the estimate of the generalized partial autocorrelation function (GPAC), which has been used for the identification of autoregressive moving‐average (ARMA) models, has a thick‐tailed asymptotic distribution. The purpose of this paper is to investigate the asymptotic behaviour of the GPAC in detail. It will be shown that the GPAC can be represented as a ratio of two functions, known as the θ function and the Λ function, each of which itself has a useful pattern for ARMA model identification. We shall show the consistencies of the extended Yule‐Walker estimates of the three functions and present their asymptotic distributions.