How large a sample is needed for the maximum likelihood estimator to be approximately Gaussian?

Abstract

The author discusses the failure of the Gaussian approximation to the distribution of the maximum-likelihood estimator in one-parameter families for finite sample sizes. Fisher (1925) has shown that this approximation is valid when an asymptotically large sample of data points is used. He did this by treating the likelihood equation (i.e. the equation obtained by setting the derivative of the likelihood function with respect to the parameter to zero) statistically and finding its solution as the sample size n is taken to infinity. The statistical treatment of the likelihood equation is extended to include corrections for finite sample sizes. The O(1/n) corrections to Fisher's asymptotic Gaussian result are calculated with corrections to the central limit theorem, and are used to derive sufficient conditions on the sample size for Fisher's result to break down. Such conditions are useful for the design of experiments. The procedure developed here can be extended to the maximum-likelihood estimation of several parameters in multivariate distributions.