Neurally inspired models of belief-propagation in arbitrary graphical models

Abstract

Listen

Translate article

In recent years psychophysical experiments have revealed information processing in the human brain to be largely consistent with Bayesian inference in graphical models. Likewise, in an attempt to unify hitherto existing approaches to learning problems, parallel ongoing research in the field of machine learning has converged to the same framework. A natural question then to ask is, what is the physical nexus between the abstract Bayesian techniques from machine learning and information processing in the brain, or, more precisely, how can the known biophysical properties of neurons and their networks give rise to such Bayesian computations? By focussing particularly on the Belief-Propagation (BP) algorithm, this thesis provides answers to that question. Our answers reside on different levels of abstraction, ranging from abstract, spike-based principles that deal with the problem at the level of neural coding, to more concrete approaches for implementing BP in neural substrates. In the first case, we define a general sampling-based scheme that entails BP as a statistical metaproperty, such that no special-purpose processing blocks for emulating the elementary BP-computations are needed. Using an interspike-interval and a spike-count code respectively, two BP processors are deduced from this scheme and verified by simulation. In particular we show that the interspike-interval processor is extremely general, since it allows for BP in arbitrary graphical models, even in presence of analog variables. Concerning the concrete BP implementations we describe and simulate a setup of interconnected Liquid-State Machines (LSMs) as an ensemble of locally interacting nodes that give rise to BP. This general message-passing architecture was inspired by the stereotypical, graph-like connectivity observed in a variety of mammalian neocortices. Moreover, the architecture is consistent with and provides a reasonable interpretation forthe hypothesized, canonical microcircuitry in cortex. Finally we combine the above two levels of abstraction, by concretely implementing the samplingbased processors in our LSM setup. With respect to the interspike-interval processor we furthermore provide an interpretation of its functionality in terms of single-neuron processing. In all cases we found our simulation results to be in very good agreement with numerical evaluations of the BP algorithm. In summary this thesis pushes forward flexible, spike-based models of BP, that reside on several levels of abstraction. At each such level, the respective models are general enough to give rise to a multitude of more concrete versions that can be experimentally tested, and of which we verify an exemplary selection by simulation. Therefore, the validity of our general models remains, even should subsequent experimental research specifically falsify their concrete versions. Moreover, some of these models provide an algorithmic meaning to the long speculated function of cortical microcircuitry, whereas others allow for BP in analog