Very small bias observed in Bayesian estimators even for small sample sizes

Abstract

The plug-in predictor of the posterior mean of the canonical parameter minimizes the posterior mean of the Kullback-Leibler loss in the exponential family. We observe that the posterior mean under the reference prior implies an approximately unbiased estimator of a suitably chosen form of the parameter. In fact, amounts of bias are very small, even for small sample sizes. While special attention is placed on the case of the inverse Gaussian distribution, other familiar distributions are examined as well. Our results suggest that the posterior mean of the canonical parameter of an exponential family under the reference prior is a promising estimator.