How regular are conjugate exponential families?

Abstract

Given an exponential family of sampling distributions of order k, one may construct in a natural way an exponential family of conjugate (that is, prior) distributions depending on a k-dimensional parameter c and an additional weight w > 0. We compute the bias term by which the expectation of the sampling mean-value parameter under the conjugate distribution deviates from the conjugate parameter c. This bias term vanishes for regular exponential families, providing an appealing interpretation of the conjugate parameter c as a ‘prior location’ of the sampling mean-value parameter. By way of example we explore the extension of this approach to moments of higher order, in order to interprete the conjugate weight w as a ‘prior sample size’.