On the adaptive linear estimators, using biased Cramér–Rao bound

Abstract

The idea of minimizing the variance in biased estimation along with controlling the gradient of bias is well established for the case of singular Fisher information matrix (FIM) in order to find the biased estimators. In this paper, the biased Cramér–Rao lower bound (BCRLB) is used to derive and study the estimate of unknown parameters in a linear model with a known twice differentiable additive noise probability density function (PDF). Even if the additive noise is not Gaussian, we show that the derived linear estimators (not unique) are linear functions of the observations (where a constant number is inserted into observation vector) in a particular form. Examples are included to illustrate the estimators performances. We show that a biased estimator obtained by optimization of BCRLB is not necessary satisfactory in a general case; therefore, additional considerations must be taken into account when using this approach. For the case where the PDF of the additive noise is not differentiable, such as uniformly distributed or time invariant magnitude noises, an asymptotical approach is given to find the estimators. As an example, we evaluate the performance of the derived adaptive filter for a first-order Markov time varying system. If the FIM is singular, we use the method of singular value decomposition (SVD) to extract the parameter estimate of the linear models. For example we show that in a linear model, parameter estimation based on single observation leads to the normalized least mean square (NLMS) algorithm. In this example using BCRLB optimization, we find the relation between the step-size of the NLMS algorithm and the bound of the bias gradient matrix.