Bayesian Estimation of Mixing Distributions

Abstract

Let $\mathscr{Q} = \{Q(t): 0 \leqq t \leqq 1\}$ be a family of probability distributions on the positive integers parameterized on [0, 1], that is \begin{equation*}\tag{1} P_t(X = x) = q_x(t);\quad x = 1,2,\cdots.\end{equation*} If $G$ is a distribution on $\lbrack 0, 1\rbrack$ the distribution of $X$ is a $G$-mixture over $\mathscr{Q}$ if \begin{equation*}\tag{2} P_G(X = x) = \int^1_0 q_x(t) dG(t) = q_x(G).\end{equation*} $G$ is called the mixing distribution. It is assumed at the outset that the family $\mathscr{Q}$ is known to be identifiable, that is if $q_x(G_1) = q_x(G_2)$ for $x = 1,2,\cdots$, then $G_1 = G_2$. See [12] and [13] for conditions insuring identifiability. Thus it makes sense to attempt to estimate $G$ when one has independent observations on $X$. Some work on estimating $G$ has been done when the mixture is finite [4], [2], [9] and for special $\mathscr{Q}$'s [6], [14]. The problem is of interest not only in an estimation context, but also in the construction of empirical Bayes decision procedures [9]. Our approach is to define a prior distribution on possible values of $G$ and then construct consistent Bayes estimates of $G$ from the posterior distribution. Section 2 gives the needed background on moment spaces, sets up the prior distribution and derives the posterior distribution. In Section 3, the Bayes estimates are defined while Section 4 proves the consistency of the posterior distribution and thus of the estimates. Here, Theorem 1 is not directly applicable to our problem, but is included because of its possible independent interest. Sections 5 and 6 generalize the earlier results.