Bayesian analysis with limited communication

Abstract

The i-th member of a group of m individuals (or stations) observes a random quantity X i , where X=(X 1,…,X m) has a density g( x ¦π). Each individual can report only y i = h i ( x i ), because of a limitation on the amount of information that can be communicated. Based on y = (y 1,…,y m) and a prior distribution π(θ), Bayesian inference or decision concerning θ is to be undertaken. The first version of this problem that will be studied is the ‘team’ problem, where the m individuals form a team with common prior π and the reports, y i, are the posterior distributions of each team member. We compare the optimal Bayesian posterior for this problem (π(θ ¦ y )) with previous suggestions, such as the optimal linear opinion pool. The second facet of the problem that is explored is that of choosing y to optimize the information communicated, subject to a constraint on the amount of information that can be communicated. In particular, we will consider the dichotomous case, in which each y i can be only 0 or 1, and will illustrate the optimal choice of y i for both inference and decision criteria. The inference criterion considered will be closeness of the posteriors π(θ ¦ x ) and π(θ ¦ y ), in an expected Kullback-Leibler sense, while the decision criterion considered will be usual optimality with respect to overall expected loss. Examples are presented, including discussion of a situation that arises in reliability demonstration.