Bias approximation for maximum likelihood estimates

Abstract

A second order approximation of the bias in maximum likelihood estimates of multiple parameters is presented. It is based on likelihood theory and Taylor's series. A single parameter approximation is commonly presented in advanced books on statistical theory; however, the corresponding multi-parameter approximation is not usually reported, especially in a matrix format. A matrix version is developed here that provides a basis for constructing relatively simple algorithms to compute the bias in maximum likelihood problems. Examples involving a nonlinear exponential model and the generalized linear model are presented.