On bias in maximum likelihood estimators

Abstract

Abstract It is well known that maximum likelihood estimators are often biased, and it is of use to estimate the expected bias so that we can reduce the mean square errors of our parameter estimates. Expressions for estimating the bias in maximum likelihood estimates have been given by Cox and Hinkley (1974) , (Theoretical Statistics, Chapman & Hall, London). In this paper, a new simple expression is derived for estimating the first-order bias in maximum likelihood estimates. This is then applied to certain specific models; namely, estimating the bias in the concentration parameter in the von Mises–Fisher distribution – for which there are some ad hoc results in the literature ( Best and Fisher, 1981 , Comm. Statist. Ser. B. Simulation Comput. 31, 493–502) – and for the scale parameter in the Cauchy distribution. In both problems, the first-order bias is found to be linear in the parameter and the sample size. Simulations are used to verify our theoretical results and to give some indication of how large the parameter and the sample size need to be for our estimates of bias to hold well.