The Bias of Autoregressive Coefficient Estimators

Abstract

Abstract This article presents simple expressions for the bias of estimators of the coefficients of an autoregressive model of arbitrary, but known, finite order. The results include models both with and without a constant term. The effects of overspecification of the model order on the bias are described. The emphasis is on least-squares and Yule-Walker estimators, but the methods extend to other estimators of similar design. Although only the order T -1 component of the bias is captured, where T is the series length, this asymptotic approximation is shown to be very accurate for least-squares estimators through some numerical simulations. The simulations examine fourth-order autoregressions chosen to resemble some data series from the literature. The order T -1 bias approximations for Yule-Walker estimators need not be accurate, especially if the zeros of the associated polynomial have moduli near 1. Examples are given where the approximation is accurate and where it is useless. The bias expressions are very simple in the case of least squares, being linear combinations of the unknown true coefficients. No interaction among the coefficients occurs. For example, if the data are a time series from a fourth-order autoregressive model with coefficients (α1, α2, α3, α4) and no constant term, the order T -1 bias of the least-squares coefficient estimator is (- α1, 1 −2α2 - α4, α1 −4α3, 1 −5α4)/T. The results differ slightly for a model with a constant term. An easily programmed algorithm for generating these expressions for any finite-order autoregressive model is given, with or without a constant term. The structure of the order T -1 bias for Yule-Walker estimators is not so readily represented, but it is easily evaluated for any model. Thus one can quickly incorporate these results into the study of other time-series problems, such as the effects of estimation error on the mean squared prediction error. Direct methods of analysis are employed to obtain the expressions. By analysis of bias in the frequency domain, infinite series representations that obscure the simple form of the bias are avoided. The key results are several lemmas regarding sums of elements within the inverse covariance matrix of p consecutive observations from an autoregression of order p. The bias approximations follow directly from these lemmas. The derivations are straightforward and yield useful insight into the structure of the estimators.