Statistical mechanics and visual signal processing

Abstract

The nervous system solves a wide variety of problems in signal processing. In many cases the performance of the nervous system is so good that it apporaches fundamental physical limits, such as the limits imposed by diffraction and photon shot noise in vision. In this paper we show how to use the language of statistical field theory to address and solve problems in signal processing, that is problems in which one must estimate some aspect of the environment from the data in an array of sensors. In the field theory formulation the optimal estimator can be written as an expectation value in an ensemble where the input data act as external field. Problems at low signal-to-noise ratio can be solved in perturbation theory, while high signal-to-noise ratios are treated with a saddle-point approximation. These ideas are illustrated in detail by an example of visual motion estimation which is chosen to model a problem solved by the fly's brain. In this problem the optimal estimator has a rich structure, adapting to various parameters of the environment such as the mean-square contrast and the correlation time of contrast fluctuations. This structure is in qualitative accord with existing measurements on motion sensitive neurons in the fly's brain, and we argue that the adaptive properties of the optimal estimator may help resolve conlficts among different interpretations of these data. Finally we propose some crucial direct tests of the adaptive behavior.