On Finite Sample Theory for Autoregressive Model Order Selection

Abstract

In the autoregressive modeling of stationary sto- chastic processes, three basic problems can be distinguished: estimation of the parameters, selection of the model order, and estimation of the expected fit of a selected model to future data. The asymptotic theory doesn't take into account the distinction that exists in practice between the different estimation meth- ods. Therefore, it is not accurate for finite samples, and a spe- cial finite sample theory is required. A comprehensive descrip- tion of the peculiarities of estimation in finite and small samples of autoregressive processes is presented. The empirical theo- retical formulas maintain as much as possible one unifying con- cept behind all different parameter estimation techniques. A finite sample criterion for order selection evolves as a natural result of our theory. This criterion serves at the same time as an improved measure for the model fit to future data. For most existing order-selection criteria equivalents are given, based on the finite sample theory. Their performance is better for small samples, and they converge to existing criteria for increasing sample sizes.