Preliminary results on relative performance of expected and observed fisher information

Abstract

Covariance calculations and confidence intervals for maximum likelihood estimates (MLEs) are commonly used in identification and statistical inference. To accurately construct such confidence intervals, one typically needs to know the covariance of the MLE. Standard statistical theory shows that the normalized MLE is asymptotically normally distributed with mean zero and covariance being the inverse of the Fisher Information Matrix (FIM) at the unknown parameter. Two common estimates for the covariance of MLE are the inverse of the observed FIM (the same as the Hessian of negative log-likelihood) and the inverse of the expected FIM (the same as FIM): both of which are evaluated at the MLE from the sample data. We show that, under reasonable conditions, the expected FIM outperforms the observed FIM under a mean-squared error criterion. This result suggests that, under certain conditions, the expected FIM is a better estimate for the covariance of MLE when used in confidence interval calculations.