MSE Bounds With Affine Bias Dominating the CramÉr–Rao Bound

Abstract

In continuation to an earlier work, we further develop bounds on the mean-squared error (MSE) when estimating a deterministic parameter vector thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> in a given estimation problem, as well as estimators that achieve the optimal performance. The traditional Cramer-Rao (CR) type bounds provide benchmarks on the variance of any estimator of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> under suitable regularity conditions, while requiring <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">priori</i> specification of a desired bias gradient. To circumvent the need to choose the bias, which is impractical in many applications, it was suggested in our earlier work to directly treat the MSE, which is the sum of the variance and the squared-norm of the bias. While previously we developed MSE bounds assuming a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linear</i> bias vector, here we study, in the same spirit, <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">affine</i> bias vectors. We demonstrate through several examples that allowing for an affine transformation can often improve the performance significantly over a linear approach. Using convex optimization tools we show that in many cases we can choose an affine bias that results in an MSE bound that is smaller than the unbiased CR bound for all values of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> . Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing an affine transformation of the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than ML for all possible values of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> .