Sequential Estimation and Diffusion of Information Over Networks: A Bayesian Approach With Exponential Family of Distributions

Abstract

Diffusion networks where nodes collaboratively estimate the parameters of stochastic models from shared observations and other estimates have become an established research topic. In this paper the problem of sequential estimation where information in the network diffuses with time is formulated abstractly and independently from any particular model. The objective is to reach a generic solution that is applicable to a wide class of popular models and based on the exponential family of distributions. The adopted Bayesian and information-theoretic paradigms provide probabilistically consistent means for incorporation of shared observations in the implemented estimation of the unknowns by the nodes as well as for effective combination of the “knowledge” of the nodes over the network. It is shown and illustrated on four examples that under certain conditions, the resulting algorithms are analytically tractable, either directly or after simple approximations. The examples include linear regression, Kalman filtering, logistic regression, and inference of an inhomogeneous Poisson process. The first two examples have their more or less direct counterparts in the state-of-the-art diffusion literature whereas the latter two are new.