Extensions of the conjugate prior through the Kullback–Leibler separators

Abstract

The conjugate prior for the exponential family, referred to also as the natural conjugate prior, is represented in terms of the Kullback–Leibler separator. This representation permits us to extend the conjugate prior to that for a general family of sampling distributions. Further, by replacing the Kullback–Leibler separator with its dual form, we define another form of a prior, which will be called the mean conjugate prior. Various results on duality between the two conjugate priors are shown. Implications of this approach include richer families of prior distributions induced by a sampling distribution and the empirical Bayes estimation of a high-dimensional mean parameter.