Rates of Convergence for an Adaptive Filtering Algorithm Driven by Stationary Dependent Data

Abstract

Eweda and Macchi [IEEE Trans. Automat. Control, 29 (1984), pp. 119–127] and Watanabe [IEEE Trans. Inform. Theory, 30 (1984), pp. 134–140] show that thee sequence of random vectors generated by a stochastic gradient adaptive filtering algorithm converges almost surely and in $L_p $ (for p an even integer) to the solution of the associated Wiener–Hopf equation when the driving data process is stationary and weakly dependent. Under strong (i.e., Rosenblatt or $\alpha $) and $\psi $ -mixing conditions, together with various moment bounds on the driving data process, an almost sure functional invariance principle is obtained that approximates the sample paths of the random process generated by the stochastic gradient algorithm with the sample paths of a particular Gauss–Markov process. Almost sure rates of convergence in the form of laws of the iterated logarithm follow from the functional invariance principle. As a byproduct a functional central limit theorem is also obtained for a sequence of processes derived by suitably scaling the sequence of iterations generated by the algorithm.