On the Skewness of the LMS Adaptive Weights

Abstract

The adjustable weights of adaptive filtering algorithms are usually assumed to obey a Gaussian distribution. This is somewhat natural under maximal-entropy considerations, since most analyses in the open literature only take into account first-and second-order statistics. This work investigates the third-order statistical feature known as skewness of the least mean square parameters distribution. Two theoretical analyses for skewness estimation are proposed: i) one that employs the independence assumption, which states that the excitation data is statistically independent from the adaptive weights; ii) one derived from the exact expectation analysis, a method that is able to predict the learning capabilities of the least mean square algorithm even when the step size is not infinitesimally small. This brief shows that the skewness of the adaptive weights distribution may present a large deviation from the common Gaussian assumption, especially in the first phase of the learning. Furthermore, it is also demonstrated that the skewness may grow without limit even when adaptive weights present convergence in both average and mean square behaviors.