Adaptive Filtering: a Theoretical and Simulation Study of the Convergence of the Parameter Estimates

Abstract

The existence of bias in the final parameter estimates using adaptive filtering is demonstrated theoretically. For observations generated by an autoregressive model of order one, an approximate theoretical expression for the bias is derived which is valid for long series of observations. The validity of the expression is investigated by simulation and comparing the theoretical bias with the simulated bias at the end of one major iteration. By carrying out a number of major iterations, it is shown that the bias reaches a stable value which is a function of the learning constant. The magnitude of the stable bias may be made as small as desired by taking smaller values for the learning constant. However, as the learning constant decreases, the number of major iterations required to achieve stability increases. By means of simulation experiments, the existence of bias in the final parameter estimates is demonstrated for shorter series of observations generated by an AR(1) model, and for long series of observations generated by an AR(2) model. Again the bias appears to increase with the magnitude of the learning constant. It is argued that the presence of this bias need not be a drawback in the practical application of the method since, by systematic reduction of the training constant between major iterations, the bias may also be reduced, while reasonably rapid convergence can still be maintained.