Encoding fidelity of binocular receptive fields with internal noise in the presence of external variability from natural scenes

Abstract

Characterizing the properties of receptive fields that are involved in encoding behaviorally-relevant latent variables (binocular disparity, motion) is a focus of visual neuroscience research. The factors that determine which receptive field properties are most useful for encoding a particular latent variable in natural scenes are not well-understood. Here, using a stereo-image database with groundtruth disparity information at each pixel and a nonlinear-linear-nonlinear subunit modeling framework, we examine how external variability and internal noise jointly determine the binocular receptive field properties that are most useful for encoding binocular disparity in natural scenes. We compute the encoding fidelity, as indexed by Fisher information, of binocular receptive field pairs across preferred spatial frequency (SF) and differences in preferred phase disparity (binocular phase shift). Two major findings emerge. First, stimulus disparity and external variability (image and depth variability) determine the usefulness of different encoding subspaces, as defined by the preferred SFs of the binocular receptive field pairs. The most useful preferred SF decreases with stimulus disparity but increases with local depth variability. Second, internal noise determines the usefulness within the encoding subspaces of different basis elements, as defined by the binocular phase shifts of the receptive fields. In particular, with internal noise, encoding fidelity varies with the particular receptive field pairs within the subspace. Non-orthogonal receptive fields are most useful, a result that deviates from the quadrature-pairs posited by the disparity energy model. Most subunit modeling frameworks do not inject noise into the individual subunit responses. This modeling choice renders all subunit receptive field pairs spanning a subspace computationally equivalent. Incorporating subunit response noise makes more specific predictions regarding the particular receptive fields that are most useful for encoding a particular latent variable. The current modeling efforts bring us closer to understanding the principles governing the design of real neural systems in natural scenes.