Geometry of the Cramer-Rao bound

Abstract

The Fisher information matrix determines how much information a measurement brings about the parameters that index the underlying probability distribution for the measurement. In this paper we assume that the parameters structure the mean value vector in a multivariate normal distribution. The Fisher matrix is then a Gramian constructed from the sensitivity vectors that characterize the first-order variation in the mean with respect to the parameters. The inverse of the Fisher matrix lower bounds the covariance of any unbiased estimator of the parameters. This inverse has several geometrical properties that bring insight into the problem of identifying multiple parameters. For example, it is the angle between a given sensitivity vector and the linear subspace spanned by all other sensitivity vectors that determines the variance bound for identifying a given parameter. Similarly, the covariance for identifying the linear influence of two different subsets of parameters depends on the principal angles between the linear subspaces spanned by the sensitivity vectors for the respective subsets. These geometrical interpretations may also be given filtering interpretations in a related multivariate normal estimation problem.