Bayesian step least mean squares algorithm for Gaussian signals

Abstract

Selecting the step of the least mean squares (LMS) algorithm is an old problem. This study uses a new approach to address this problem resulting in a new algorithm with excellent system identification performance. The LMS algorithm, with time-varying step, size can be shown to be equivalent to the Kalman filter in some conditions. This is as long as the state noise of the Kalman filter and the step size of the LMS algorithm are chosen carefully. The Kalman filter is the optimum linear estimator (Bayesian) given the state and the measurement noise covariance matrices, but these matrices are not always known. This work considers the case where these matrices are not known, in the special cases that the Kalman filter reduces to the LMS. This results in an algorithm to select the step-size of the LMS algorithm with few priors. The optimum step size can be calculated using estimates of the probability density function (PDF) of the coefficient estimation error variance (qw ) and measurement noise variance (qv ). The PDFs can be estimated from the data using Bayes’ rule and assuming Gaussian reference and measurement noise signals. The resulting algorithm to determine qw and qv is a second small Kalman filter, and the outputs of this filter (means and covariances) are used to determine the expected value of the step.