Stochastic pooling networks

Abstract

We introduce and define the concept of a stochastic pooling network (SPN), as a model forsensor systems where redundancy and two forms of ‘noise’—lossy compression and randomness—interact in surprising ways. Our approach to analysing SPNs is informationtheoretic. We define an SPN as a network with multiple nodes that each produce noisy andcompressed measurements of the same information. An SPN must combine all thesemeasurements into a single further compressed network output, in a way dictated solely bynaturally occurring physical properties—i.e. pooling—and yet cause no (or negligible)reduction in mutual information. This means that SPNs exhibit redundancy reduction asan emergent property of pooling. The SPN concept is applicable to examples inbiological neural coding, nanoelectronics, distributed sensor networks, digitalbeamforming arrays, image processing, multiaccess communication networks and socialnetworks. In most cases the randomness is assumed to be unavoidably presentrather than deliberately introduced. We illustrate the central properties of SPNsfor several case studies, where pooling occurs by summation, including nodesthat are noisy scalar quantizers, and nodes with conditionally Poisson statistics.Other emergent properties of SPNs and some unsolved problems are also brieflydiscussed.